This is an automated email from the ASF dual-hosted git repository.
sergehuber pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/unomi.git
The following commit(s) were added to refs/heads/master by this push:
new 360a4d86c UNOMI-968: Pluggable persistence IT harness for ES, OS, and
other backends (#833)
360a4d86c is described below
commit 360a4d86ce1ba8cd1f5ab042f1ef0c26433973af
Author: Serge Huber <[email protected]>
AuthorDate: Tue Jul 21 11:47:11 2026 +0200
UNOMI-968: Pluggable persistence IT harness for ES, OS, and other backends
(#833)
Merge pluggeable persistence integration tests modifications
---
itests/README.md | 96 ++++++-
itests/pom.xml | 32 +++
.../test/java/org/apache/unomi/itests/AllITs.java | 4 +-
.../test/java/org/apache/unomi/itests/BaseIT.java | 309 +++++++++++----------
.../{AllITs.java => CorePersistenceITs.java} | 11 +-
.../org/apache/unomi/itests/HealthCheckIT.java | 51 +++-
.../java/org/apache/unomi/itests/JSONSchemaIT.java | 11 +-
.../org/apache/unomi/itests/ProfileServiceIT.java | 6 +-
.../itests/ProfileServiceWithoutOverwriteIT.java | 3 +-
.../org/apache/unomi/itests/ProgressListener.java | 123 ++++----
.../java/org/apache/unomi/itests/RolloverIT.java | 42 ++-
.../org/apache/unomi/itests/TestTimingCache.java | 143 ++++++++--
.../apache/unomi/itests/TestTimingCacheTest.java | 123 ++++++++
.../migration/Migrate16xToCurrentVersionIT.java | 45 +--
.../itests/persistence/ElasticsearchITBackend.java | 117 ++++++++
.../itests/persistence/OpenSearchITBackend.java | 130 +++++++++
.../itests/persistence/PersistenceITBackend.java | 92 ++++++
.../persistence/PersistenceITBackendResolver.java | 110 ++++++++
.../persistence/PersistenceITCapabilities.java | 204 ++++++++++++++
.../unomi/itests/persistence/SearchBackendIT.java | 29 ++
...e.unomi.itests.persistence.PersistenceITBackend | 19 ++
21 files changed, 1425 insertions(+), 275 deletions(-)
diff --git a/itests/README.md b/itests/README.md
index ebb3d9708..0d587d2b4 100644
--- a/itests/README.md
+++ b/itests/README.md
@@ -32,7 +32,9 @@ and data migration.
`docker-maven-plugin`).
- **Maven Failsafe** runs a single entry point — `AllITs` — which aggregates
all test
classes. Each test class extends `BaseIT`, which handles Karaf startup, OSGi
service
- injection, and common test utilities.
+ injection, and common test utilities. Persistence is selected via
+ `unomi.persistence.provider` (Elasticsearch / OpenSearch today; see
+ [Pluggable persistence providers](#pluggable-persistence-providers)).
A full IT run typically takes 20–30 minutes. The Karaf instance is created
fresh for each
run under `itests/target/exam/` with a UUID directory name.
@@ -284,6 +286,98 @@ See the [Maven Failsafe plugin
docs](https://maven.apache.org/surefire/maven-fai
---
+## Pluggable persistence providers
+
+Unomi’s runtime persistence is an OSGi capability
(`unomi.persistence;provider:=…`).
+The IT harness follows the same idea: `BaseIT` resolves a test-only
+`PersistenceITBackend` instead of hard-coding Elasticsearch / OpenSearch.
+
+### Provider selection
+
+| Property | Role |
+|----------|------|
+| `unomi.persistence.provider` | Preferred provider id (`elasticsearch`,
`opensearch`, or an extension id) |
+| `unomi.search.engine` | **Deprecated alias** — used when
`unomi.persistence.provider` is unset |
+| `unomi.persistence.it.backend` | Optional FQCN of a `PersistenceITBackend`
implementation |
+
+Default CI cells set both `unomi.persistence.provider` and the deprecated
+`unomi.search.engine` so existing `--use-opensearch` / scripts keep working.
+
+### Progress / ETA (local timing cache)
+
+`ProgressSuite` + `ProgressListener` write a best-effort per-test duration
cache under
+the `itests` module directory (survives `mvn clean`):
+
+`.test-timing-cache-<provider>.properties`
+
+One file per persistence provider (`elasticsearch`, `opensearch`,
`postgresql`, …)
+so ETAs are not mixed across backends. On later runs the listener sums
remaining
+historical times and scales them by how fast/slow the current run is vs history
+(clamped). Safe to delete; missing/unwritable cache falls back to in-run
averages.
+
+### Built-in backends
+
+| Id | Class | Suite for CI |
+|----|-------|--------------|
+| `elasticsearch` | `ElasticsearchITBackend` | `AllITs` (default) |
+| `opensearch` | `OpenSearchITBackend` | `AllITs` with `-Duse.opensearch=true`
|
+
+### Search-only vs core behavioural tests
+
+`AllITs` and `CorePersistenceITs` share the **same** test membership for
maximum coverage.
+Tests that need HTTP admin / snapshot / rollover APIs stay in the suite and use
+`Assume` + `PersistenceITCapabilities` so unsupported backends **skip**
(reported as
+skipped, not green false-pass):
+
+| Class | Gate |
+|-------|------|
+| `Migrate16xToCurrentVersionIT` | `snapshotRestoreMigration()` |
+| `RolloverIT` | `indexRolloverApi().isPresent()` + `httpAdminApi()`; switch
on `LIFECYCLE` / `STATE_MANAGEMENT` |
+| `HealthCheckIT` | Always asserts `karaf` / `unomi` / `persistence`; optional
`providerNamedHealthProbe` / `clusterHealthProbe` |
+
+`HealthCheckIT` is not search-only: it runs on every provider.
+
+Elasticsearch / OpenSearch CI continues to use `AllITs`. Portable / non-search
cells
+(PostgreSQL, …) should run `CorePersistenceITs` (identical membership, clearer
name).
+
+```bash
+mvn clean install -P integration-tests
-Dit.test=org.apache.unomi.itests.CorePersistenceITs
+```
+
+Capability flags gate remaining special-cases
+(e.g. `snapshotRestoreMigration`, `flattenedRangeQueryResult`,
`indexRolloverApi`)
+so providers skip what they cannot support instead of checking product names.
+
+### Adding a third-party backend
+
+1. Implement `org.apache.unomi.itests.persistence.PersistenceITBackend`
+ (feature options, distribution name for `unomi:setup`, ConfigAdmin PID,
+ capabilities, await-ready / HTTP helpers as needed).
+2. Put the implementation on the Failsafe test classpath.
+3. Register it either:
+ - in
`META-INF/services/org.apache.unomi.itests.persistence.PersistenceITBackend`, or
+ - via `-Dunomi.persistence.it.backend=com.example.MyPersistenceITBackend`
+4. Select it with `-Dunomi.persistence.provider=<your-id>`.
+5. Run `CorePersistenceITs` (same membership as `AllITs`). Incomplete SPI
+ implementations may still fail behavioural tests — unsupported HTTP-admin
+ features are skipped via `Assume`, not removed from the suite.
+
+`unomi-itests` publishes a **test-jar** (`mvn -Pintegration-tests -pl itests
install`) so external modules can
+depend on `BaseIT`, `CorePersistenceITs`, and `PersistenceITBackend` without
forking
+sources. Optionally implement `prepareBeforeUnomiSetup` to patch ConfigAdmin
(e.g. JDBC
+DataSource) after `UnomiManagementService` is up and before `unomi:setup`.
+
+Example Failsafe exclude when running `AllITs` against a non-search provider:
+
+```xml
+<excludedGroups>org.apache.unomi.itests.persistence.SearchBackendIT</excludedGroups>
+```
+
+(Note: JUnit 4 category filtering applies to classes Failsafe launches
directly;
+prefer `CorePersistenceITs` when using the suite entry point.)
+
+---
+
## Debugging Integration Tests
### Attaching a remote debugger to the test JVM
diff --git a/itests/pom.xml b/itests/pom.xml
index d28e1d2a6..026a1798f 100644
--- a/itests/pom.xml
+++ b/itests/pom.xml
@@ -28,6 +28,7 @@
<description>Apache Unomi Context Server integration tests</description>
<properties>
+ <unomi.persistence.provider>elasticsearch</unomi.persistence.provider>
<unomi.search.engine>elasticsearch</unomi.search.engine>
<use.opensearch>false</use.opensearch>
<docker.container.name>itests-opensearch</docker.container.name>
@@ -277,6 +278,33 @@
</systemPropertyVariables>
</configuration>
</plugin>
+ <!-- Publish test classes so out-of-tree PersistenceService
providers can implement
+ PersistenceITBackend and reuse CorePersistenceITs / BaseIT
(UNOMI-968). -->
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>test-jar</id>
+ <goals>
+ <goal>test-jar</goal>
+ </goals>
+ <configuration>
+ <!-- Include IT classes and test resources
(fixtures used by BaseIT.config). -->
+ <includes>
+ <include>org/apache/unomi/itests/**</include>
+ <include>**/*.cfg</include>
+ <include>**/*.csv</include>
+ <include>**/*.json</include>
+ <include>**/*.groovy</include>
+ <include>migration/**</include>
+ <include>schemas/**</include>
+ <include>META-INF/**</include>
+ </includes>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
</plugins>
</build>
@@ -306,6 +334,8 @@
</includes>
<systemPropertyVariables>
<my.system.property>foo</my.system.property>
+
<unomi.persistence.provider>elasticsearch</unomi.persistence.provider>
+ <!-- Deprecated alias; keep for older scripts
-->
<unomi.search.engine>elasticsearch</unomi.search.engine>
<elasticsearch.port>${elasticsearch.port}</elasticsearch.port>
<it.karaf.heap>${karaf.heap}</it.karaf.heap>
@@ -432,6 +462,8 @@
</includes>
<systemPropertyVariables>
<my.system.property>foo</my.system.property>
+
<unomi.persistence.provider>opensearch</unomi.persistence.provider>
+ <!-- Deprecated alias; keep for older scripts
-->
<unomi.search.engine>opensearch</unomi.search.engine>
<org.apache.unomi.opensearch.addresses>localhost:${opensearch.port}</org.apache.unomi.opensearch.addresses>
<org.ops4j.pax.logging.DefaultServiceLog.level>INFO</org.ops4j.pax.logging.DefaultServiceLog.level>
diff --git a/itests/src/test/java/org/apache/unomi/itests/AllITs.java
b/itests/src/test/java/org/apache/unomi/itests/AllITs.java
index 8617b1dcb..41351e5b0 100644
--- a/itests/src/test/java/org/apache/unomi/itests/AllITs.java
+++ b/itests/src/test/java/org/apache/unomi/itests/AllITs.java
@@ -24,7 +24,9 @@ import org.junit.runner.RunWith;
import org.junit.runners.Suite.SuiteClasses;
/**
- * Defines suite of test classes to run.
+ * Defines suite of test classes to run (Elasticsearch / OpenSearch CI
default).
+ * Same membership as {@link CorePersistenceITs}; use capabilities + {@code
Assume} for
+ * provider-specific skips rather than maintaining a smaller suite.
*
* @author Sergiy Shyrkov
*/
diff --git a/itests/src/test/java/org/apache/unomi/itests/BaseIT.java
b/itests/src/test/java/org/apache/unomi/itests/BaseIT.java
index 8587f4f8a..5ec6385ba 100644
--- a/itests/src/test/java/org/apache/unomi/itests/BaseIT.java
+++ b/itests/src/test/java/org/apache/unomi/itests/BaseIT.java
@@ -22,8 +22,6 @@ import org.apache.camel.CamelContext;
import org.apache.camel.Route;
import org.apache.camel.ServiceStatus;
import org.apache.commons.io.IOUtils;
-import org.apache.http.auth.AuthScope;
-import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.HttpEntity;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.*;
@@ -54,6 +52,9 @@ import org.apache.unomi.api.tenants.Tenant;
import org.apache.unomi.api.tenants.TenantService;
import org.apache.unomi.api.utils.ConditionBuilder;
import org.apache.unomi.groovy.actions.services.GroovyActionsService;
+import org.apache.unomi.itests.persistence.PersistenceITBackend;
+import org.apache.unomi.itests.persistence.PersistenceITBackendResolver;
+import org.apache.unomi.itests.persistence.PersistenceITCapabilities;
import org.apache.unomi.itests.tools.LogChecker;
import org.apache.unomi.itests.tools.httpclient.HttpClientThatWaitsForUnomi;
import org.apache.unomi.lifecycle.BundleWatcher;
@@ -98,9 +99,11 @@ import javax.net.ssl.X509TrustManager;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
+import java.net.URL;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
+import java.nio.file.StandardCopyOption;
import java.security.KeyManagementException;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
@@ -135,16 +138,23 @@ public abstract class BaseIT extends KarafTestSupport {
protected static final int DEFAULT_TRYING_TRIES = 10;
protected static final int DEFAULT_SHOULDBETRUE_TRIES = 5;
- protected static final String SEARCH_ENGINE_PROPERTY =
"unomi.search.engine";
+ /** @deprecated use {@link
PersistenceITBackendResolver#PROVIDER_PROPERTY}. */
+ @Deprecated
+ protected static final String SEARCH_ENGINE_PROPERTY =
PersistenceITBackendResolver.SEARCH_ENGINE_PROPERTY;
protected static final String SEARCH_ENGINE_HTTPREQUEST_LOG_LEVEL =
"unomi.search.engine.httprequest.log.level";
- protected static final String SEARCH_ENGINE_ELASTICSEARCH =
"elasticsearch";
- protected static final String SEARCH_ENGINE_OPENSEARCH = "opensearch";
+ protected static final String SEARCH_ENGINE_ELASTICSEARCH =
PersistenceITBackendResolver.PROVIDER_ELASTICSEARCH;
+ protected static final String SEARCH_ENGINE_OPENSEARCH =
PersistenceITBackendResolver.PROVIDER_OPENSEARCH;
+ protected static final String PERSISTENCE_PROVIDER_PROPERTY =
PersistenceITBackendResolver.PROVIDER_PROPERTY;
protected static final String RESOLVER_DEBUG_PROPERTY =
"it.unomi.resolver.debug";
protected static final String ENABLE_LOG_CHECKING_PROPERTY =
"it.unomi.log.checking.enabled";
protected static final String CAMEL_DEBUG_PROPERTY =
"it.unomi.camel.debug";
protected static boolean unomiStarted = false;
+ /**
+ * Active provider id. Kept for existing tests; prefer {@link
#getPersistenceBackend()}.
+ */
protected static String searchEngine = SEARCH_ENGINE_ELASTICSEARCH;
+ private static PersistenceITBackend persistenceBackend;
private static boolean searchEngineConfiguredForTesting = false;
private static boolean searchEngineHealthVerifiedAfterStartup = false;
@@ -239,37 +249,71 @@ public abstract class BaseIT extends KarafTestSupport {
}
protected void checkSearchEngine() {
- searchEngine = System.getProperty(SEARCH_ENGINE_PROPERTY,
SEARCH_ENGINE_ELASTICSEARCH);
+ resolvePersistenceBackend();
configureSearchEngineForTesting();
}
+ /**
+ * Resolves (and caches) the active {@link PersistenceITBackend} from
system properties /
+ * ServiceLoader. Also updates the legacy {@link #searchEngine} field.
+ */
+ protected static PersistenceITBackend resolvePersistenceBackend() {
+ if (persistenceBackend == null) {
+ persistenceBackend = PersistenceITBackendResolver.resolve();
+ searchEngine = persistenceBackend.providerId();
+ }
+ return persistenceBackend;
+ }
+
+ protected static PersistenceITBackend getPersistenceBackend() {
+ return resolvePersistenceBackend();
+ }
+
+ protected PersistenceITCapabilities persistenceCapabilities() {
+ return getPersistenceBackend().capabilities();
+ }
+
+ /** ConfigAdmin PID for the active persistence provider (e.g.
throwExceptions). */
+ protected String persistenceConfigPid() {
+ return getPersistenceBackend().persistenceConfigPid();
+ }
+
@Before
public void waitForStartup() throws InterruptedException {
// disable retry
retry = new KarafTestSupport.Retry(false);
- // Check search engine and apply any necessary fixes (e.g.,
default_template deletion)
+ // Resolve provider and apply any necessary backend prep (e.g.,
zero-replica template)
checkSearchEngine();
+ try {
+ getPersistenceBackend().awaitBackendReady();
+ } catch (InterruptedException e) {
+ Thread.currentThread().interrupt();
+ throw e;
+ } catch (Exception e) {
+ throw new IllegalStateException("Persistence backend
awaitBackendReady failed", e);
+ }
// Start Unomi if not already done
if (!unomiStarted) {
// We must check that the Unomi Management Service is up and
running before launching the
// command otherwise the start configuration will not be properly
populated.
waitForUnomiManagementService();
- if (SEARCH_ENGINE_ELASTICSEARCH.equals(searchEngine)) {
- LOGGER.info("Starting Unomi with elasticsearch search
engine...");
- System.out.println("==== Starting Unomi with elasticsearch
search engine...");
- executeCommand("unomi:setup
-d=unomi-distribution-elasticsearch -f=true");
- executeCommand("unomi:start");
- } else if (SEARCH_ENGINE_OPENSEARCH.equals(searchEngine)){
- LOGGER.info("Starting Unomi with opensearch search engine...");
- System.out.println("==== Starting Unomi with opensearch search
engine...");
- executeCommand("unomi:setup -d=unomi-distribution-opensearch
-f=true");
- executeCommand("unomi:start");
- } else {
- LOGGER.error("Unknown search engine: " + searchEngine);
- throw new InterruptedException("Unknown search engine: " +
searchEngine);
+ PersistenceITBackend backend = getPersistenceBackend();
+ try {
+ backend.prepareBeforeUnomiSetup(bundleContext,
configurationAdmin);
+ } catch (InterruptedException e) {
+ Thread.currentThread().interrupt();
+ throw e;
+ } catch (Exception e) {
+ throw new IllegalStateException("Persistence backend
prepareBeforeUnomiSetup failed", e);
}
+ String distribution = backend.distributionFeature();
+ LOGGER.info("Starting Unomi with persistence provider {}
(distribution {})...", backend.providerId(), distribution);
+ System.out.println("==== Starting Unomi with persistence provider
" + backend.providerId()
+ + " (distribution " + distribution + ")...");
+ executeCommand("unomi:setup -d=" + distribution + " -f=true");
+ executeCommand("unomi:start");
unomiStarted = true;
}
@@ -542,6 +586,43 @@ public abstract class BaseIT extends KarafTestSupport {
Thread.sleep(1000);
}
+ /**
+ * Resolves a fixture under {@code src/test/resources} for Pax Exam {@code
replaceConfigurationFile}.
+ * Prefers the local filesystem (Unomi itests module); falls back to the
classpath so out-of-tree
+ * consumers of the {@code unomi-itests} test-jar can reuse {@link
#config()} without copying files.
+ */
+ protected static File resolveTestResource(String
pathUnderSrcTestResources) {
+ File local = new File(pathUnderSrcTestResources);
+ if (local.isFile()) {
+ return local;
+ }
+ String classpathName = pathUnderSrcTestResources;
+ final String prefix = "src/test/resources/";
+ if (classpathName.startsWith(prefix)) {
+ classpathName = classpathName.substring(prefix.length());
+ }
+ URL url = BaseIT.class.getClassLoader().getResource(classpathName);
+ if (url == null) {
+ throw new IllegalStateException("Missing test resource: " +
pathUnderSrcTestResources
+ + " (not on filesystem or classpath)");
+ }
+ try {
+ if ("file".equals(url.getProtocol())) {
+ return new File(url.toURI());
+ }
+ String fileName = new File(classpathName).getName();
+ Path tmp = Files.createTempFile("unomi-it-resource-", "-" +
fileName);
+ try (InputStream in = url.openStream()) {
+ Files.copy(in, tmp, StandardCopyOption.REPLACE_EXISTING);
+ }
+ File out = tmp.toFile();
+ out.deleteOnExit();
+ return out;
+ } catch (Exception e) {
+ throw new IllegalStateException("Cannot resolve test resource " +
pathUnderSrcTestResources, e);
+ }
+ }
+
@Override
public MavenArtifactUrlReference getKarafDistribution() {
return
maven().groupId("org.apache.unomi").artifactId("unomi").versionAsInProject().type("tar.gz");
@@ -554,120 +635,41 @@ public abstract class BaseIT extends KarafTestSupport {
System.out.println("==== Configuring container");
}
- searchEngine = System.getProperty(SEARCH_ENGINE_PROPERTY,
SEARCH_ENGINE_ELASTICSEARCH);
+ resolvePersistenceBackend();
if (!configLogged) {
- LOGGER.info("Search Engine: {}", searchEngine);
- System.out.println("Search Engine: " + searchEngine);
- }
-
- // Define features option based on search engine
- Option featuresOption;
- Option distributionOption;
- if (SEARCH_ENGINE_ELASTICSEARCH.equals(searchEngine)) {
- featuresOption = features(
-
maven().groupId("org.apache.unomi").artifactId("unomi-kar").versionAsInProject().type("xml").classifier("features"),
- "unomi-base",
- "unomi-startup",
- "unomi-elasticsearch-core",
- "unomi-persistence-core",
- "unomi-services",
- "unomi-cxs-privacy-extension-services",
- "unomi-plugins-base",
- "unomi-plugins-request",
- "unomi-plugins-mail",
- "unomi-plugins-optimization-test",
- "unomi-rest-api",
- "unomi-cxs-privacy-extension",
- "unomi-elasticsearch-conditions",
- "unomi-cxs-lists-extension",
- "unomi-cxs-geonames-extension",
- "unomi-shell-dev-commands",
- "unomi-wab",
- "unomi-web-tracker",
- "unomi-healthcheck-elasticsearch",
- "unomi-router-karaf-feature",
- "unomi-groovy-actions",
- "unomi-rest-ui",
- "cdp-graphql-feature",
- "unomi-startup-complete"
- );
- distributionOption = features(
-
maven().groupId("org.apache.unomi").artifactId("unomi-distribution").versionAsInProject().type("xml").classifier("features"),
- "unomi-distribution-elasticsearch-graphql"
- );
- } else if (SEARCH_ENGINE_OPENSEARCH.equals(searchEngine)) {
- featuresOption = features(
-
maven().groupId("org.apache.unomi").artifactId("unomi-kar").versionAsInProject().type("xml").classifier("features"),
- "unomi-base",
- "unomi-startup",
- "unomi-opensearch-core",
- "unomi-persistence-core",
- "unomi-services",
- "unomi-cxs-privacy-extension-services",
- "unomi-plugins-base",
- "unomi-plugins-request",
- "unomi-plugins-mail",
- "unomi-plugins-optimization-test",
- "unomi-rest-api",
- "unomi-cxs-privacy-extension",
- "unomi-opensearch-conditions",
- "unomi-cxs-lists-extension",
- "unomi-cxs-geonames-extension",
- "unomi-shell-dev-commands",
- "unomi-wab",
- "unomi-web-tracker",
- "unomi-healthcheck-opensearch",
- "unomi-router-karaf-feature",
- "unomi-groovy-actions",
- "unomi-rest-ui",
- "cdp-graphql-feature",
- "unomi-startup-complete"
- );
- distributionOption = features(
-
maven().groupId("org.apache.unomi").artifactId("unomi-distribution").versionAsInProject().type("xml").classifier("features"),
- "unomi-distribution-opensearch-graphql"
- );
- } else {
- throw new IllegalArgumentException("Unknown search engine: " +
searchEngine);
+ LOGGER.info("Persistence provider: {}", searchEngine);
+ System.out.println("Persistence provider: " + searchEngine);
}
+ PersistenceITBackend backend = getPersistenceBackend();
+ Option[] backendFeatures = backend.featureOptions();
+ Option[] backendConfig = backend.configurationOptions();
+
Option[] options = new Option[]{
- replaceConfigurationFile("etc/org.apache.unomi.router.cfg",
new File("src/test/resources/org.apache.unomi.router.cfg")),
- replaceConfigurationFile("data/tmp/1-basic-test.csv", new
File("src/test/resources/1-basic-test.csv")),
-
replaceConfigurationFile("data/tmp/recurrent_import/2-surfers-test.csv", new
File("src/test/resources/2-surfers-test.csv")),
-
replaceConfigurationFile("data/tmp/recurrent_import/3-surfers-overwrite-test.csv",
new File("src/test/resources/3-surfers-overwrite-test.csv")),
-
replaceConfigurationFile("data/tmp/recurrent_import/4-surfers-delete-test.csv",
new File("src/test/resources/4-surfers-delete-test.csv")),
-
replaceConfigurationFile("data/tmp/recurrent_import/5-ranking-test.csv", new
File("src/test/resources/5-ranking-test.csv")),
-
replaceConfigurationFile("data/tmp/recurrent_import/6-actors-test.csv", new
File("src/test/resources/6-actors-test.csv")),
- replaceConfigurationFile("data/tmp/testLogin.json", new
File("src/test/resources/testLogin.json")),
- replaceConfigurationFile("data/tmp/testCopyProperties.json",
new File("src/test/resources/testCopyProperties.json")),
-
replaceConfigurationFile("data/tmp/testCopyPropertiesWithoutSystemTags.json",
new File("src/test/resources/testCopyPropertiesWithoutSystemTags.json")),
-
replaceConfigurationFile("data/tmp/testLoginEventCondition.json", new
File("src/test/resources/testLoginEventCondition.json")),
-
replaceConfigurationFile("data/tmp/testClickEventCondition.json", new
File("src/test/resources/testClickEventCondition.json")),
- replaceConfigurationFile("data/tmp/testRuleGroovyAction.json",
new File("src/test/resources/testRuleGroovyAction.json")),
-
replaceConfigurationFile("data/tmp/conditions/testIdsConditionLegacy.json", new
File("src/test/resources/conditions/testIdsConditionLegacy.json")),
-
replaceConfigurationFile("data/tmp/conditions/testIdsConditionNew.json", new
File("src/test/resources/conditions/testIdsConditionNew.json")),
-
replaceConfigurationFile("data/tmp/conditions/testBooleanConditionLegacy.json",
new File("src/test/resources/conditions/testBooleanConditionLegacy.json")),
-
replaceConfigurationFile("data/tmp/conditions/testPropertyConditionLegacy.json",
new File("src/test/resources/conditions/testPropertyConditionLegacy.json")),
-
replaceConfigurationFile("data/tmp/groovy/UpdateAddressAction.groovy", new
File("src/test/resources/groovy/UpdateAddressAction.groovy")),
+ replaceConfigurationFile("etc/org.apache.unomi.router.cfg",
resolveTestResource("src/test/resources/org.apache.unomi.router.cfg")),
+ replaceConfigurationFile("data/tmp/1-basic-test.csv",
resolveTestResource("src/test/resources/1-basic-test.csv")),
+
replaceConfigurationFile("data/tmp/recurrent_import/2-surfers-test.csv",
resolveTestResource("src/test/resources/2-surfers-test.csv")),
+
replaceConfigurationFile("data/tmp/recurrent_import/3-surfers-overwrite-test.csv",
resolveTestResource("src/test/resources/3-surfers-overwrite-test.csv")),
+
replaceConfigurationFile("data/tmp/recurrent_import/4-surfers-delete-test.csv",
resolveTestResource("src/test/resources/4-surfers-delete-test.csv")),
+
replaceConfigurationFile("data/tmp/recurrent_import/5-ranking-test.csv",
resolveTestResource("src/test/resources/5-ranking-test.csv")),
+
replaceConfigurationFile("data/tmp/recurrent_import/6-actors-test.csv",
resolveTestResource("src/test/resources/6-actors-test.csv")),
+ replaceConfigurationFile("data/tmp/testLogin.json",
resolveTestResource("src/test/resources/testLogin.json")),
+ replaceConfigurationFile("data/tmp/testCopyProperties.json",
resolveTestResource("src/test/resources/testCopyProperties.json")),
+
replaceConfigurationFile("data/tmp/testCopyPropertiesWithoutSystemTags.json",
resolveTestResource("src/test/resources/testCopyPropertiesWithoutSystemTags.json")),
+
replaceConfigurationFile("data/tmp/testLoginEventCondition.json",
resolveTestResource("src/test/resources/testLoginEventCondition.json")),
+
replaceConfigurationFile("data/tmp/testClickEventCondition.json",
resolveTestResource("src/test/resources/testClickEventCondition.json")),
+ replaceConfigurationFile("data/tmp/testRuleGroovyAction.json",
resolveTestResource("src/test/resources/testRuleGroovyAction.json")),
+
replaceConfigurationFile("data/tmp/conditions/testIdsConditionLegacy.json",
resolveTestResource("src/test/resources/conditions/testIdsConditionLegacy.json")),
+
replaceConfigurationFile("data/tmp/conditions/testIdsConditionNew.json",
resolveTestResource("src/test/resources/conditions/testIdsConditionNew.json")),
+
replaceConfigurationFile("data/tmp/conditions/testBooleanConditionLegacy.json",
resolveTestResource("src/test/resources/conditions/testBooleanConditionLegacy.json")),
+
replaceConfigurationFile("data/tmp/conditions/testPropertyConditionLegacy.json",
resolveTestResource("src/test/resources/conditions/testPropertyConditionLegacy.json")),
+
replaceConfigurationFile("data/tmp/groovy/UpdateAddressAction.groovy",
resolveTestResource("src/test/resources/groovy/UpdateAddressAction.groovy")),
editConfigurationFilePut("etc/org.ops4j.pax.logging.cfg",
"log4j2.rootLogger.level", "INFO"),
editConfigurationFilePut("etc/org.apache.karaf.features.cfg",
"serviceRequirements", "disable"),
editConfigurationFilePut("etc/system.properties",
"my.system.property", System.getProperty("my.system.property")),
- editConfigurationFilePut("etc/system.properties",
SEARCH_ENGINE_PROPERTY, System.getProperty(SEARCH_ENGINE_PROPERTY,
SEARCH_ENGINE_ELASTICSEARCH)),
- editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.elasticsearch.cluster.name", "contextElasticSearchITests"),
- editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.elasticsearch.addresses", "localhost:" + getSearchPort()),
- editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.elasticsearch.taskWaitingPollingInterval", "50"),
- editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.elasticsearch.rollover.maxDocs", "300"),
- editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.elasticsearch.minimalClusterState", "YELLOW"),
- editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.opensearch.cluster.name", "contextElasticSearchITests"),
- editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.opensearch.addresses", "localhost:" + getSearchPort()),
- editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.opensearch.username", "admin"),
- editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.opensearch.password", "Unomi.1ntegrat10n.Tests"),
- editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.opensearch.sslEnable", "false"),
- editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.opensearch.sslTrustAllCertificates", "true"),
- editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.opensearch.rollover.maxDocs", "300"),
- editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.opensearch.minimalClusterState", "YELLOW"),
+ editConfigurationFilePut("etc/system.properties",
SEARCH_ENGINE_PROPERTY, searchEngine),
+ editConfigurationFilePut("etc/system.properties",
PERSISTENCE_PROVIDER_PROPERTY, searchEngine),
editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.migration.tenant.id", TEST_TENANT_ID),
// Default scheduler.thread.poolSize (5) is sized for the
near-instant in-memory unit-test
// double, not a real ES/OS backend. Under real refresh/write
latency, the checker, task
@@ -680,9 +682,6 @@ public abstract class BaseIT extends KarafTestSupport {
systemProperty("org.ops4j.pax.exam.rbc.rmi.port").value("1199"),
systemProperty("org.apache.unomi.healthcheck.enabled").value("true"),
- featuresOption, // Add the features option
- distributionOption, // Add the distribution option
-
configureConsole().startRemoteShell(),
logLevel(LogLevel.INFO),
keepRuntimeFolder(),
@@ -691,6 +690,8 @@ public abstract class BaseIT extends KarafTestSupport {
};
List<Option> karafOptions = new ArrayList<>();
karafOptions.addAll(Arrays.asList(options));
+ karafOptions.addAll(Arrays.asList(backendConfig));
+ karafOptions.addAll(Arrays.asList(backendFeatures));
String karafDebug = System.getProperty("it.karaf.debug");
if (karafDebug != null) {
@@ -1435,6 +1436,10 @@ public abstract class BaseIT extends KarafTestSupport {
if (searchEngineConfiguredForTesting) {
return;
}
+ if (!persistenceCapabilities().httpAdminApi()) {
+ searchEngineConfiguredForTesting = true;
+ return;
+ }
try (CloseableHttpClient client = createSearchEngineHttpClient()) {
String baseUrl = getSearchEngineBaseUrl();
ensureZeroReplicaIndexTemplate(client, baseUrl);
@@ -1464,6 +1469,9 @@ public abstract class BaseIT extends KarafTestSupport {
}
private void enforceZeroReplicasAndWaitForCluster(String context) {
+ if (!persistenceCapabilities().httpAdminApi()) {
+ return;
+ }
try (CloseableHttpClient client = createSearchEngineHttpClient()) {
String baseUrl = getSearchEngineBaseUrl();
ensureZeroReplicaIndexTemplate(client, baseUrl);
@@ -1517,6 +1525,14 @@ public abstract class BaseIT extends KarafTestSupport {
* Stage 3 (post-Unomi start): assert cluster health for single-node IT
(ES and OpenSearch).
*/
protected void assertClusterHealthy(String context) {
+ if (!persistenceCapabilities().httpAdminApi()) {
+ try {
+ getPersistenceBackend().assertHealthyAfterUnomiStart();
+ } catch (Exception e) {
+ throw new IllegalStateException("Backend health assertion
failed: " + context, e);
+ }
+ return;
+ }
try (CloseableHttpClient client = createSearchEngineHttpClient()) {
String baseUrl = getSearchEngineBaseUrl();
ensureZeroReplicaIndexTemplate(client, baseUrl);
@@ -1550,36 +1566,31 @@ public abstract class BaseIT extends KarafTestSupport {
}
protected static String getSearchEngineBaseUrl() {
- if (SEARCH_ENGINE_OPENSEARCH.equals(searchEngine)) {
- return "http://localhost:"; + getSearchPort();
- }
- return "http://localhost:"; + getSearchPort();
+ requireHttpAdminApi("getSearchEngineBaseUrl");
+ return getPersistenceBackend().searchBaseUrl();
}
protected CloseableHttpClient createSearchEngineHttpClient() throws
IOException {
- if (SEARCH_ENGINE_OPENSEARCH.equals(searchEngine)) {
- BasicCredentialsProvider credentialsProvider = new
BasicCredentialsProvider();
- credentialsProvider.setCredentials(AuthScope.ANY,
- new UsernamePasswordCredentials("admin",
"Unomi.1ntegrat10n.Tests"));
- return HttpUtils.initHttpClient(true, credentialsProvider);
- }
- return HttpUtils.initHttpClient(true, null);
+ requireHttpAdminApi("createSearchEngineHttpClient");
+ return getPersistenceBackend().createSearchHttpClient();
}
/**
- * Gets the appropriate search engine port based on the configured search
engine.
+ * Gets the appropriate search engine port based on the configured
persistence provider.
*
* @return The port number as a string
*/
protected static String getSearchPort() {
- String searchEngine = System.getProperty(SEARCH_ENGINE_PROPERTY,
SEARCH_ENGINE_ELASTICSEARCH);
- if (SEARCH_ENGINE_OPENSEARCH.equals(searchEngine)) {
- // For OpenSearch, get the port from the system property set by
maven-failsafe-plugin
- return System.getProperty("org.apache.unomi.opensearch.addresses",
"localhost:9401")
- .split(":")[1]; // Extract port number from
"localhost:9401"
- } else {
- // For Elasticsearch, use the default port or system property if
set
- return System.getProperty("elasticsearch.port", "9400");
+ requireHttpAdminApi("getSearchPort");
+ return getPersistenceBackend().searchPort();
+ }
+
+ private static void requireHttpAdminApi(String caller) {
+ PersistenceITBackend backend = getPersistenceBackend();
+ if (!backend.capabilities().httpAdminApi()) {
+ throw new UnsupportedOperationException(
+ caller + " requires httpAdminApi, but provider '" +
backend.providerId()
+ + "' does not advertise it (skip via Assume /
SearchBackendIT / CorePersistenceITs)");
}
}
diff --git a/itests/src/test/java/org/apache/unomi/itests/AllITs.java
b/itests/src/test/java/org/apache/unomi/itests/CorePersistenceITs.java
similarity index 85%
copy from itests/src/test/java/org/apache/unomi/itests/AllITs.java
copy to itests/src/test/java/org/apache/unomi/itests/CorePersistenceITs.java
index 8617b1dcb..6cc692a0d 100644
--- a/itests/src/test/java/org/apache/unomi/itests/AllITs.java
+++ b/itests/src/test/java/org/apache/unomi/itests/CorePersistenceITs.java
@@ -24,9 +24,12 @@ import org.junit.runner.RunWith;
import org.junit.runners.Suite.SuiteClasses;
/**
- * Defines suite of test classes to run.
- *
- * @author Sergiy Shyrkov
+ * Full behavioural IT suite for any {@code PersistenceService} provider.
+ * <p>
+ * Membership matches {@link AllITs}. Tests that need HTTP admin / snapshot /
rollover
+ * APIs use {@link org.junit.Assume} on {@link
org.apache.unomi.itests.persistence.PersistenceITCapabilities}
+ * so unsupported backends <em>skip</em> (not fail, not false-pass). Prefer
this suite for
+ * PostgreSQL, in-memory, JDBC, etc.; Elasticsearch / OpenSearch CI may keep
using {@link AllITs}.
*/
@RunWith(ProgressSuite.class)
@SuiteClasses({
@@ -83,5 +86,5 @@ import org.junit.runners.Suite.SuiteClasses;
RolloverIT.class,
HealthCheckIT.class
})
-public class AllITs {
+public class CorePersistenceITs {
}
diff --git a/itests/src/test/java/org/apache/unomi/itests/HealthCheckIT.java
b/itests/src/test/java/org/apache/unomi/itests/HealthCheckIT.java
index 9fe11829d..305c2f38e 100644
--- a/itests/src/test/java/org/apache/unomi/itests/HealthCheckIT.java
+++ b/itests/src/test/java/org/apache/unomi/itests/HealthCheckIT.java
@@ -41,7 +41,12 @@ import java.util.concurrent.*;
import static org.junit.Assert.fail;
/**
- * Health Check Integration Tests
+ * Health Check Integration Tests.
+ * <p>
+ * Always asserts {@code karaf}, {@code unomi}, and {@code persistence} LIVE.
+ * Optional probes ({@code providerNamedHealthProbe}, {@code
clusterHealthProbe}) are
+ * asserted only when the active {@link
org.apache.unomi.itests.persistence.PersistenceITCapabilities}
+ * advertise them — so the same class covers Elasticsearch, OpenSearch,
PostgreSQL, etc.
*/
@RunWith(PaxExam.class)
@ExamReactorStrategy(PerSuite.class)
@@ -57,15 +62,10 @@ public class HealthCheckIT extends BaseIT {
public void testHealthCheck() {
try {
List<HealthCheckResponse> response = get(HEALTHCHECK_ENDPOINT, new
TypeReference<>() {});
- LOGGER.info("configured search engine: {}", searchEngine);
+ LOGGER.info("configured persistence provider: {}",
getPersistenceBackend().providerId());
LOGGER.info("health check response: {}", response);
Assert.assertNotNull(response);
- Assert.assertEquals(5, response.size());
- Assert.assertTrue(response.stream().anyMatch(r ->
r.getName().equals("karaf") && r.getStatus() ==
HealthCheckResponse.Status.LIVE));
- Assert.assertTrue(response.stream().anyMatch(r ->
r.getName().equals(searchEngine) && r.getStatus() ==
HealthCheckResponse.Status.LIVE));
- Assert.assertTrue(response.stream().anyMatch(r ->
r.getName().equals("unomi") && r.getStatus() ==
HealthCheckResponse.Status.LIVE));
- Assert.assertTrue(response.stream().anyMatch(r ->
r.getName().equals("persistence") && r.getStatus() ==
HealthCheckResponse.Status.LIVE));
- Assert.assertTrue(response.stream().anyMatch(r ->
r.getName().equals("cluster") && r.getStatus() ==
HealthCheckResponse.Status.LIVE));
+ assertHealthCheckLive(response);
} catch (Exception e) {
LOGGER.error("Error while executing health check", e);
fail("Error while executing health check" + e.getMessage());
@@ -88,12 +88,7 @@ public class HealthCheckIT extends BaseIT {
}
for (Future<List<HealthCheckResponse>> future : futures) {
List<HealthCheckResponse> health = future.get(10,
TimeUnit.SECONDS);
- Assert.assertEquals(5, health.size());
- Assert.assertTrue(health.stream().anyMatch(r ->
r.getName().equals("karaf") && r.getStatus() ==
HealthCheckResponse.Status.LIVE));
- Assert.assertTrue(health.stream().anyMatch(r ->
r.getName().equals(searchEngine) && r.getStatus() ==
HealthCheckResponse.Status.LIVE));
- Assert.assertTrue(health.stream().anyMatch(r ->
r.getName().equals("unomi") && r.getStatus() ==
HealthCheckResponse.Status.LIVE));
- Assert.assertTrue(health.stream().anyMatch(r ->
r.getName().equals("persistence") && r.getStatus() ==
HealthCheckResponse.Status.LIVE));
- Assert.assertTrue(health.stream().anyMatch(r ->
r.getName().equals("cluster") && r.getStatus() ==
HealthCheckResponse.Status.LIVE));
+ assertHealthCheckLive(health);
}
Thread.sleep(10);
}
@@ -109,6 +104,34 @@ public class HealthCheckIT extends BaseIT {
}
}
+ private void assertHealthCheckLive(List<HealthCheckResponse> response) {
+ Assert.assertNotNull(response);
+ Assert.assertTrue(response.stream().anyMatch(r ->
r.getName().equals("karaf") && r.getStatus() ==
HealthCheckResponse.Status.LIVE));
+ Assert.assertTrue(response.stream().anyMatch(r ->
r.getName().equals("unomi") && r.getStatus() ==
HealthCheckResponse.Status.LIVE));
+ Assert.assertTrue(response.stream().anyMatch(r ->
r.getName().equals("persistence") && r.getStatus() ==
HealthCheckResponse.Status.LIVE));
+ int expectedMinProbes = 3; // karaf, unomi, persistence
+ boolean named = persistenceCapabilities().providerNamedHealthProbe();
+ boolean cluster = persistenceCapabilities().clusterHealthProbe();
+ if (named) {
+ expectedMinProbes++;
+ String namedProbe = getPersistenceBackend().providerId();
+ Assert.assertTrue(response.stream().anyMatch(r ->
r.getName().equals(namedProbe) && r.getStatus() ==
HealthCheckResponse.Status.LIVE));
+ }
+ if (cluster) {
+ expectedMinProbes++;
+ Assert.assertTrue(response.stream().anyMatch(r ->
r.getName().equals("cluster") && r.getStatus() ==
HealthCheckResponse.Status.LIVE));
+ }
+ // ES/OS advertise both optional probes → keep historical exact size
(5).
+ // Other backends may expose extra probes; require at least the
expected set.
+ if (named && cluster) {
+ Assert.assertEquals("Unexpected health probe count: " + response,
expectedMinProbes, response.size());
+ } else {
+ Assert.assertTrue(
+ "Expected at least " + expectedMinProbes + " health
probes, got " + response.size() + ": " + response,
+ response.size() >= expectedMinProbes);
+ }
+ }
+
protected <T> T get(final String url, TypeReference<T> typeReference) {
CloseableHttpResponse response = null;
try {
diff --git a/itests/src/test/java/org/apache/unomi/itests/JSONSchemaIT.java
b/itests/src/test/java/org/apache/unomi/itests/JSONSchemaIT.java
index e37c23cc8..e9fcd7c37 100644
--- a/itests/src/test/java/org/apache/unomi/itests/JSONSchemaIT.java
+++ b/itests/src/test/java/org/apache/unomi/itests/JSONSchemaIT.java
@@ -25,6 +25,7 @@ import org.apache.http.util.EntityUtils;
import org.apache.unomi.api.Event;
import org.apache.unomi.api.Scope;
import org.apache.unomi.api.conditions.Condition;
+import org.apache.unomi.itests.persistence.PersistenceITCapabilities;
import org.apache.unomi.itests.tools.LogChecker;
import org.apache.unomi.itests.tools.httpclient.HttpClientThatWaitsForUnomi;
import org.apache.unomi.schema.api.JsonSchemaWrapper;
@@ -377,16 +378,14 @@ public class JSONSchemaIT extends BaseIT {
// Refresh to ensure event is queryable
refreshPersistence(Event.class);
final Condition finalCondition = condition;
- // For Elasticsearch, range queries on flattened properties should
return null or empty list
- // For OpenSearch, they may return results
+ // Providers differ on range queries against flattened properties
(hits vs empty).
// We just need to wait for the query to execute (not throw an
exception)
refreshPersistence(Event.class);
org.apache.unomi.api.PartialList<Event> queryResult =
persistenceService.query(finalCondition, null, Event.class, 0, -1);
- if ("opensearch".equals(searchEngine)) {
- assertNotNull("OpenSearch should return results for flattened
properties", queryResult);
+ if (persistenceCapabilities().flattenedRangeQueryResult() ==
PersistenceITCapabilities.FlattenedRangeQueryResult.HITS) {
+ assertNotNull("provider that returns flattened range hits should
return results", queryResult);
} else {
- // Elasticsearch should return null or empty list for range
queries on flattened properties
- assertTrue("Elasticsearch should return null or empty list for
flattened properties range query",
+ assertTrue("provider that returns empty for flattened range should
return null or empty list",
queryResult == null || queryResult.getList() == null ||
queryResult.getList().isEmpty());
}
diff --git a/itests/src/test/java/org/apache/unomi/itests/ProfileServiceIT.java
b/itests/src/test/java/org/apache/unomi/itests/ProfileServiceIT.java
index dff72a1dc..e256c7004 100644
--- a/itests/src/test/java/org/apache/unomi/itests/ProfileServiceIT.java
+++ b/itests/src/test/java/org/apache/unomi/itests/ProfileServiceIT.java
@@ -153,7 +153,7 @@ public class ProfileServiceIT extends BaseIT {
@Test
public void testGetProfileWithWrongScrollerIdThrowException() throws
InterruptedException, IOException {
boolean throwExceptionCurrent = false;
- Configuration searchEngineConfiguration =
configurationAdmin.getConfiguration("org.apache.unomi.persistence." +
searchEngine);
+ Configuration searchEngineConfiguration =
configurationAdmin.getConfiguration(persistenceConfigPid());
if (searchEngineConfiguration != null &&
searchEngineConfiguration.getProperties().get("throwExceptions") != null) {
try {
if
(searchEngineConfiguration.getProperties().get("throwExceptions") instanceof
String) {
@@ -167,7 +167,7 @@ public class ProfileServiceIT extends BaseIT {
}
}
- updateConfiguration(null, "org.apache.unomi.persistence." +
searchEngine, "throwExceptions", true);
+ updateConfiguration(null, persistenceConfigPid(), "throwExceptions",
true);
Query query = new Query();
query.setLimit(2);
@@ -180,7 +180,7 @@ public class ProfileServiceIT extends BaseIT {
} catch (RuntimeException ex) {
// Should get here since this scenario should throw exception
} finally {
- updateConfiguration(null, "org.apache.unomi.persistence." +
searchEngine, "throwExceptions",
+ updateConfiguration(null, persistenceConfigPid(),
"throwExceptions",
throwExceptionCurrent);
}
}
diff --git
a/itests/src/test/java/org/apache/unomi/itests/ProfileServiceWithoutOverwriteIT.java
b/itests/src/test/java/org/apache/unomi/itests/ProfileServiceWithoutOverwriteIT.java
index 2f94dd198..922b08bf9 100644
---
a/itests/src/test/java/org/apache/unomi/itests/ProfileServiceWithoutOverwriteIT.java
+++
b/itests/src/test/java/org/apache/unomi/itests/ProfileServiceWithoutOverwriteIT.java
@@ -44,8 +44,7 @@ public class ProfileServiceWithoutOverwriteIT extends BaseIT {
@Configuration
public Option[] config() {
-
- searchEngine = System.getProperty(SEARCH_ENGINE_PROPERTY,
SEARCH_ENGINE_ELASTICSEARCH);
+ resolvePersistenceBackend();
System.out.println("Search Engine: " + searchEngine);
List<Option> options = new ArrayList<>();
diff --git a/itests/src/test/java/org/apache/unomi/itests/ProgressListener.java
b/itests/src/test/java/org/apache/unomi/itests/ProgressListener.java
index 78d9653b1..1ecdca270 100644
--- a/itests/src/test/java/org/apache/unomi/itests/ProgressListener.java
+++ b/itests/src/test/java/org/apache/unomi/itests/ProgressListener.java
@@ -16,6 +16,7 @@
*/
package org.apache.unomi.itests;
+import org.apache.unomi.itests.persistence.PersistenceITBackendResolver;
import org.junit.runner.Description;
import org.junit.runner.Result;
import org.junit.runner.notification.Failure;
@@ -32,6 +33,7 @@ import java.util.Map;
import java.util.PriorityQueue;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.CopyOnWriteArrayList;
import java.util.concurrent.atomic.AtomicInteger;
/**
@@ -44,7 +46,7 @@ import java.util.concurrent.atomic.AtomicInteger;
* <li>ASCII art logo display at test suite startup</li>
* <li>Real-time progress bar with percentage completion</li>
* <li>Colorized output (when ANSI is supported)</li>
- * <li>Estimated time remaining calculations</li>
+ * <li>Estimated time remaining from a per-persistence-provider historical
timing cache</li>
* <li>Test success/failure counters</li>
* <li>Top 10 slowest tests tracking and reporting</li>
* <li>Motivational quotes displayed at progress milestones</li>
@@ -137,15 +139,24 @@ public class ProgressListener extends RunListener {
private long startTime = System.currentTimeMillis();
/** Timestamp when the current individual test started */
private long startTestTime = System.currentTimeMillis();
+ /**
+ * Set in {@link #testFailure} before {@link #testFinished}; failed tests
must not update the
+ * timing cache (aborted / assertion failures skew historical ETAs).
+ */
+ private boolean currentTestFailed;
/** Formatter for human-readable timestamps */
private static final DateTimeFormatter TIMESTAMP_FORMATTER =
DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
- /** Search engine under test (e.g. "elasticsearch", "opensearch"); timings
are cached per engine */
- private final String searchEngine =
System.getProperty(BaseIT.SEARCH_ENGINE_PROPERTY,
BaseIT.SEARCH_ENGINE_ELASTICSEARCH);
- /** Cached per-test durations loaded from {@link TestTimingCache} */
+ /** Persistence provider under test (e.g. elasticsearch, opensearch,
postgresql); timings cached per provider */
+ private final String persistenceProvider;
+ /** Cached per-test durations loaded from {@link TestTimingCache} for this
provider */
private final Map<String, Long> cachedTimings;
/** Timing-cache keys for tests not yet completed in this run */
private final Set<String> remainingTestKeys;
+ /** Durations (ms) of tests completed in this run */
+ private final List<Long> completedDurations = new CopyOnWriteArrayList<>();
+ /** Pairs of [observedMs, cachedMs] for completed tests that had a
historical entry */
+ private final List<long[]> observedVsCached = new CopyOnWriteArrayList<>();
/**
* Creates a new ProgressListener instance.
@@ -162,7 +173,8 @@ public class ProgressListener extends RunListener {
this.completedTests = completedTests;
this.slowTests = new PriorityQueue<>((t1, t2) -> Long.compare(t1.time,
t2.time));
this.ansiSupported = isAnsiSupported();
- this.cachedTimings = TestTimingCache.load(searchEngine);
+ this.persistenceProvider =
PersistenceITBackendResolver.resolveProviderId();
+ this.cachedTimings = TestTimingCache.load(persistenceProvider);
this.remainingTestKeys = ConcurrentHashMap.newKeySet();
this.remainingTestKeys.addAll(testKeys);
}
@@ -259,11 +271,12 @@ public class ProgressListener extends RunListener {
// Print the bottom border
System.out.println(colorize(bottomBorder, CYAN));
- // Display search engine information once at the start
- String searchEngine = System.getProperty("unomi.search.engine",
"elasticsearch");
- String searchEngineDisplay = capitalizeSearchEngine(searchEngine);
+ // Display persistence provider + historical timing cache info once at
the start
+ String providerDisplay = capitalizeProvider(persistenceProvider);
System.out.println();
- System.out.println(colorize("Using search engine: " +
searchEngineDisplay, CYAN));
+ System.out.println(colorize("Persistence provider: " +
providerDisplay, CYAN));
+ System.out.println(colorize("Historical timings: " +
cachedTimings.size() + "/" + totalTests
+ + " tests cached → " +
TestTimingCache.cacheFile(persistenceProvider).toAbsolutePath(), CYAN));
System.out.println();
}
@@ -274,6 +287,7 @@ public class ProgressListener extends RunListener {
*/
@Override
public void testStarted(Description description) {
+ currentTestFailed = false;
startTestTime = System.currentTimeMillis();
// Print test start boundary with test name
String testName = extractTestName(description);
@@ -287,7 +301,9 @@ public class ProgressListener extends RunListener {
}
/**
- * Called when an individual test finishes successfully. Updates counters
and displays progress.
+ * Called when an individual test finishes. Updates counters and displays
progress.
+ * Successful tests are written to {@link TestTimingCache} immediately so
a killed mid-suite
+ * run still retains timings for every test that completed cleanly.
*
* @param description the description of the test that finished
*/
@@ -295,6 +311,9 @@ public class ProgressListener extends RunListener {
public void testFinished(Description description) {
long endTestTime = System.currentTimeMillis();
long testDuration = endTestTime - startTestTime;
+ boolean failed = currentTestFailed;
+ currentTestFailed = false;
+
completedTests.incrementAndGet();
successfulTests.incrementAndGet(); // Default to success unless a
failure is recorded separately.
slowTests.add(new TestTime(description.getDisplayName(),
testDuration));
@@ -304,10 +323,18 @@ public class ProgressListener extends RunListener {
}
String testKey = TestTimingCache.keyFor(description);
remainingTestKeys.remove(testKey);
- // Persist immediately (rather than batching until testRunFinished) so
a run that gets killed
- // mid-suite (Ctrl-C, CI timeout, a hung test force-killed) still
leaves every test that did
- // complete recorded in the cache for next time.
- TestTimingCache.save(searchEngine, Collections.singletonMap(testKey,
testDuration));
+
+ // Persist only successes: failure/abort durations pollute the
provider cache and ETA scale.
+ // Write after every successful test (not only at suite end) so Ctrl-C
/ CI kill keeps progress.
+ if (!failed) {
+ completedDurations.add(testDuration);
+ Long historical = cachedTimings.get(testKey);
+ if (historical != null && historical > 0L) {
+ observedVsCached.add(new long[]{testDuration, historical});
+ }
+ TestTimingCache.save(persistenceProvider,
Collections.singletonMap(testKey, testDuration));
+ }
+
// Print test end boundary
String testName = extractTestName(description);
String durationStr = formatTime(testDuration);
@@ -323,12 +350,25 @@ public class ProgressListener extends RunListener {
}
/**
- * Called when a test fails. Updates failure counters and displays the
failure message.
+ * {@code @Ignore}d tests never call {@link #testFinished}; drop them from
the remaining set so ETA
+ * does not keep budgeting time for them.
+ */
+ @Override
+ public void testIgnored(Description description) {
+ remainingTestKeys.remove(TestTimingCache.keyFor(description));
+ completedTests.incrementAndGet();
+ displayProgress();
+ }
+
+ /**
+ * Called when a test fails (before {@link #testFinished}). Marks the test
so its duration is
+ * not written to the timing cache.
*
* @param failure the failure information
*/
@Override
public void testFailure(Failure failure) {
+ currentTestFailed = true;
successfulTests.decrementAndGet(); // Remove the previous success
count for this test.
failedTests.incrementAndGet();
String testName = extractTestName(failure.getDescription());
@@ -391,26 +431,25 @@ public class ProgressListener extends RunListener {
}
/**
- * Capitalizes the search engine name for display.
- * Converts "opensearch" to "OpenSearch" and "elasticsearch" to
"Elasticsearch".
+ * Capitalizes the persistence provider name for display.
*
- * @param searchEngine the search engine name (lowercase)
- * @return the capitalized search engine name
+ * @param provider the provider id (lowercase)
+ * @return a display-friendly name
*/
- private String capitalizeSearchEngine(String searchEngine) {
- if (searchEngine == null || searchEngine.isEmpty()) {
- return searchEngine;
+ private String capitalizeProvider(String provider) {
+ if (provider == null || provider.isEmpty()) {
+ return provider;
}
- // Handle special case for "opensearch" -> "OpenSearch"
- if ("opensearch".equalsIgnoreCase(searchEngine)) {
+ if ("opensearch".equalsIgnoreCase(provider)) {
return "OpenSearch";
}
- // Handle "elasticsearch" -> "Elasticsearch"
- if ("elasticsearch".equalsIgnoreCase(searchEngine)) {
+ if ("elasticsearch".equalsIgnoreCase(provider)) {
return "Elasticsearch";
}
- // Default: capitalize first letter
- return searchEngine.substring(0, 1).toUpperCase() +
searchEngine.substring(1);
+ if ("postgresql".equalsIgnoreCase(provider)) {
+ return "PostgreSQL";
+ }
+ return provider.substring(0, 1).toUpperCase() + provider.substring(1);
}
/**
@@ -454,32 +493,20 @@ public class ProgressListener extends RunListener {
}
/**
- * Estimates the remaining time for the run by summing, for each test that
has not completed yet,
- * its historical duration from the search-engine-specific {@link
TestTimingCache} when one exists,
- * and falling back to this run's own flat average for any test with no
cache entry (e.g. the very
- * first run on a machine, or a newly added test).
+ * Estimates remaining time using the provider-specific {@link
TestTimingCache}, scaled by how
+ * fast/slow this run has been vs history for tests that already completed
with a cache hit.
*
* @param completed the number of tests completed so far
* @param elapsedTime the time elapsed since the run started, in
milliseconds
* @return the estimated remaining time, in milliseconds
*/
private long estimateRemainingTime(int completed, long elapsedTime) {
- // Avoid division by very low completed count; use a floor value
- int stableCompleted = Math.max(completed, 1);
- double averageTestTimeMillis = elapsedTime / (double) stableCompleted;
-
- long estimate = 0;
- int uncachedRemaining = 0;
- for (String key : remainingTestKeys) {
- Long cached = cachedTimings.get(key);
- if (cached != null) {
- estimate += cached;
- } else {
- uncachedRemaining++;
- }
- }
- estimate += (long) (averageTestTimeMillis * uncachedRemaining);
- return estimate;
+ return TestTimingCache.estimateRemainingMs(
+ remainingTestKeys,
+ cachedTimings,
+ observedVsCached,
+ completedDurations,
+ elapsedTime);
}
/**
diff --git a/itests/src/test/java/org/apache/unomi/itests/RolloverIT.java
b/itests/src/test/java/org/apache/unomi/itests/RolloverIT.java
index 449bfa278..667070dde 100644
--- a/itests/src/test/java/org/apache/unomi/itests/RolloverIT.java
+++ b/itests/src/test/java/org/apache/unomi/itests/RolloverIT.java
@@ -18,8 +18,12 @@ package org.apache.unomi.itests;
import com.fasterxml.jackson.databind.JsonNode;
import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.unomi.itests.persistence.PersistenceITCapabilities;
+import org.apache.unomi.itests.persistence.SearchBackendIT;
import org.apache.unomi.shell.migration.utils.HttpUtils;
+import org.junit.Assume;
import org.junit.Test;
+import org.junit.experimental.categories.Category;
import org.junit.runner.RunWith;
import org.ops4j.pax.exam.junit.PaxExam;
import org.ops4j.pax.exam.spi.reactors.ExamReactorStrategy;
@@ -31,6 +35,7 @@ import java.util.Map;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
/**
* Verifies that Unomi correctly wires up the event index's rollover lifecycle
on both Elasticsearch (ILM)
@@ -52,9 +57,14 @@ import static org.junit.Assert.assertTrue;
* that happens ISM has nothing left to transition to (the rollover policy's
single state has no further
* transitions) and marks that now-rolled-over index's management as
completed/disabled, which would make a
* hardcoded context-event-000001 check fail even though the rollover
machinery worked correctly.
+ * <p>
+ * Non-search providers (e.g. PostgreSQL) leave {@link
PersistenceITCapabilities.IndexRolloverApi#NONE}
+ * so this test is {@link Assume Assumed} skipped (not failed). Kept in {@link
CorePersistenceITs}
+ * / {@link AllITs} for maximum suite coverage across backends.
*/
@RunWith(PaxExam.class)
@ExamReactorStrategy(PerSuite.class)
+@Category(SearchBackendIT.class)
public class RolloverIT extends BaseIT {
private static final String EVENT_ALIAS = "context-event";
@@ -63,16 +73,36 @@ public class RolloverIT extends BaseIT {
@Test
public void testEventIndexRolloverIsProperlyConfigured() throws Exception {
+ PersistenceITCapabilities caps = persistenceCapabilities();
+ Assume.assumeTrue(
+ "Index rollover requires an HTTP rollover API (provider="
+ + getPersistenceBackend().providerId() + ")",
+ caps.indexRolloverApi().isPresent());
+ Assume.assumeTrue(
+ "Index rollover assertions require httpAdminApi (provider="
+ + getPersistenceBackend().providerId() + ")",
+ caps.httpAdminApi());
+
try (CloseableHttpClient client = createSearchEngineHttpClient()) {
String writeIndex = resolveCurrentWriteIndex(client);
JsonNode indexRoot = getJson(client, "/" + writeIndex +
"/_settings?flat_settings=true").get(writeIndex);
assertTrue("Expected the event write index " + writeIndex + " to
already exist", indexRoot != null);
JsonNode settings = indexRoot.get("settings");
- if (SEARCH_ENGINE_OPENSEARCH.equals(searchEngine)) {
- assertOpenSearchRolloverConfigured(client, settings,
writeIndex);
- } else {
- assertElasticsearchRolloverConfigured(client, settings,
writeIndex);
+ switch (caps.indexRolloverApi()) {
+ case STATE_MANAGEMENT:
+ assertStateManagementRolloverConfigured(client, settings,
writeIndex);
+ break;
+ case LIFECYCLE:
+ assertLifecycleRolloverConfigured(client, settings,
writeIndex);
+ break;
+ case NONE:
+ fail("unreachable: indexRolloverApi was assumed present");
+ break;
+ default: {
+ PersistenceITCapabilities.IndexRolloverApi unexpected =
caps.indexRolloverApi();
+ throw new IllegalStateException("Unhandled
IndexRolloverApi: " + unexpected);
+ }
}
}
}
@@ -94,7 +124,7 @@ public class RolloverIT extends BaseIT {
throw new AssertionError("Could not find a write index for alias " +
EVENT_ALIAS);
}
- private void assertOpenSearchRolloverConfigured(CloseableHttpClient
client, JsonNode settings, String writeIndex) throws IOException {
+ private void assertStateManagementRolloverConfigured(CloseableHttpClient
client, JsonNode settings, String writeIndex) throws IOException {
assertEquals("event index should reference the Unomi rollover policy",
POLICY_ID, text(settings,
"index.plugins.index_state_management.policy_id"));
assertEquals("event index rollover_alias should be the event write
alias",
@@ -116,7 +146,7 @@ public class RolloverIT extends BaseIT {
assertEquals(EXPECTED_MAX_DOCS,
rolloverAction.get("min_doc_count").asLong());
}
- private void assertElasticsearchRolloverConfigured(CloseableHttpClient
client, JsonNode settings, String writeIndex) throws IOException {
+ private void assertLifecycleRolloverConfigured(CloseableHttpClient client,
JsonNode settings, String writeIndex) throws IOException {
assertEquals("event index should reference the Unomi rollover policy",
POLICY_ID, text(settings, "index.lifecycle.name"));
assertEquals("event index rollover_alias should be the event write
alias",
diff --git a/itests/src/test/java/org/apache/unomi/itests/TestTimingCache.java
b/itests/src/test/java/org/apache/unomi/itests/TestTimingCache.java
index 99e05e802..f6e5b7e55 100644
--- a/itests/src/test/java/org/apache/unomi/itests/TestTimingCache.java
+++ b/itests/src/test/java/org/apache/unomi/itests/TestTimingCache.java
@@ -28,6 +28,7 @@ import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardCopyOption;
+import java.util.Collection;
import java.util.HashMap;
import java.util.Locale;
import java.util.Map;
@@ -37,13 +38,14 @@ import java.util.Properties;
* Local, best-effort cache of individual IT execution times, used by {@link
ProgressListener} to make its
* estimated-time-remaining calculation more accurate than a single flat
running average.
* <p>
- * The cache is a plain properties file kept in the {@code itests} module
directory (not {@code target/}) so
- * it survives {@code mvn clean}, with one file per search engine since
Elasticsearch and OpenSearch runs have
- * different timing profiles and shouldn't be averaged together.
+ * The cache is a plain properties file kept under {@code user.dir} (typically
the {@code itests} module
+ * directory, not {@code target/}) so it survives {@code mvn clean}, with
<strong>one file per
+ * persistence provider</strong> ({@code elasticsearch}, {@code opensearch},
{@code postgresql}, …)
+ * since backends have different timing profiles and must not be averaged
together.
* <p>
* This is a local developer convenience, not build state: all I/O failures
are swallowed so a missing or
- * unwritable cache (e.g. a read-only or ephemeral CI workspace) never fails
the IT run - it just falls back
- * to the flat average for every test, matching the prior behavior.
+ * unwritable cache (e.g. a read-only or ephemeral CI workspace) never fails
the IT run — it just falls back
+ * to the in-run average for every test.
*/
final class TestTimingCache {
@@ -52,6 +54,12 @@ final class TestTimingCache {
/** Weight given to a freshly observed duration when blending it into the
persisted average. */
private static final double SMOOTHING = 0.3;
+ /**
+ * Clamp for the live-run vs historical scale factor so a few outliers
cannot make ETA absurd.
+ */
+ static final double MIN_SCALE = 0.25;
+ static final double MAX_SCALE = 4.0;
+
private TestTimingCache() {
}
@@ -90,15 +98,15 @@ final class TestTimingCache {
}
/**
- * Loads the previously persisted timings for the given search engine.
+ * Loads the previously persisted timings for the given persistence
provider.
*
- * @param searchEngine the search engine the current run targets (e.g.
"elasticsearch", "opensearch")
+ * @param persistenceProvider provider id (e.g. {@code elasticsearch},
{@code opensearch}, {@code postgresql})
* @return a mutable map of cache key to last known duration in
milliseconds; empty if no cache
* exists yet or it could not be read
*/
- static Map<String, Long> load(String searchEngine) {
+ static Map<String, Long> load(String persistenceProvider) {
Map<String, Long> timings = new HashMap<>();
- Path cacheFile = cacheFile(searchEngine);
+ Path cacheFile = cacheFile(persistenceProvider);
if (!Files.isReadable(cacheFile)) {
return timings;
}
@@ -121,20 +129,20 @@ final class TestTimingCache {
}
/**
- * Merges freshly observed durations into the persisted cache for the
given search engine, smoothing
- * each updated entry with an exponential moving average so a single
unusually slow/fast run doesn't
- * swing future ETAs too far.
+ * Merges freshly observed durations into the persisted cache for the
given persistence provider,
+ * smoothing each updated entry with an exponential moving average so a
single unusually slow/fast
+ * run doesn't swing future ETAs too far.
*
- * @param searchEngine the search engine the run just executed against
+ * @param persistenceProvider provider id the run just executed against
* @param observedTimings durations (in milliseconds) observed during the
run that just finished
*/
- static void save(String searchEngine, Map<String, Long> observedTimings) {
+ static void save(String persistenceProvider, Map<String, Long>
observedTimings) {
if (observedTimings.isEmpty()) {
return;
}
- Path cacheFile = cacheFile(searchEngine);
+ Path cacheFile = cacheFile(persistenceProvider);
try {
- Map<String, Long> merged = load(searchEngine);
+ Map<String, Long> merged = load(persistenceProvider);
for (Map.Entry<String, Long> entry : observedTimings.entrySet()) {
Long previous = merged.get(entry.getKey());
long updated = previous == null
@@ -148,7 +156,8 @@ final class TestTimingCache {
Path parent = cacheFile.toAbsolutePath().getParent();
Path tempFile = Files.createTempFile(parent, "test-timing-cache",
".tmp");
try (Writer writer = Files.newBufferedWriter(tempFile,
StandardCharsets.UTF_8)) {
- props.store(writer, "Apache Unomi IT test timing cache (local
dev aid, safe to delete)");
+ props.store(writer, "Apache Unomi IT test timing cache per
persistence provider "
+ + "(local dev aid, safe to delete)");
}
Files.move(tempFile, cacheFile,
StandardCopyOption.REPLACE_EXISTING);
} catch (IOException | RuntimeException e) {
@@ -157,10 +166,102 @@ final class TestTimingCache {
}
}
- private static Path cacheFile(String searchEngine) {
- String normalizedEngine = (searchEngine == null ||
searchEngine.isEmpty())
+ /**
+ * Estimates remaining wall time for unfinished tests.
+ * <p>
+ * For each remaining test with a historical entry, uses that duration
scaled by how fast/slow
+ * <em>this</em> run has been relative to history (ratio of observed vs
cached for completed
+ * tests that had a cache hit). Uncached remaining tests use the in-run
average of completed
+ * durations (or the median of historical values when nothing has
completed yet).
+ *
+ * @param remainingKeys keys still expected to run
+ * @param cachedTimings historical durations for this persistence provider
+ * @param observedVsCachedCompleted pairs of (observedMs, cachedMs) for
completed tests that had history
+ * @param completedDurations all completed durations this run (for
fallback average)
+ * @param elapsedTimeMs wall time since suite start (unused for sum; kept
for API clarity)
+ * @return estimated remaining milliseconds (never negative)
+ */
+ static long estimateRemainingMs(Collection<String> remainingKeys,
+ Map<String, Long> cachedTimings,
+ Collection<long[]>
observedVsCachedCompleted,
+ Collection<Long> completedDurations,
+ long elapsedTimeMs) {
+ double scale = computeScale(observedVsCachedCompleted);
+ double fallbackAvg = fallbackAverageMs(completedDurations,
cachedTimings, elapsedTimeMs);
+
+ long estimate = 0L;
+ for (String key : remainingKeys) {
+ Long cached = cachedTimings.get(key);
+ if (cached != null && cached > 0L) {
+ estimate += Math.round(cached * scale);
+ } else {
+ estimate += Math.round(fallbackAvg);
+ }
+ }
+ return Math.max(0L, estimate);
+ }
+
+ /**
+ * How fast/slow this run is vs the historical cache for the same provider.
+ * {@code 1.0} = on pace; {@code >1} = slower than history; {@code <1} =
faster.
+ */
+ static double computeScale(Collection<long[]> observedVsCachedCompleted) {
+ if (observedVsCachedCompleted == null ||
observedVsCachedCompleted.isEmpty()) {
+ return 1.0;
+ }
+ long observedSum = 0L;
+ long cachedSum = 0L;
+ for (long[] pair : observedVsCachedCompleted) {
+ if (pair == null || pair.length < 2) {
+ continue;
+ }
+ if (pair[0] > 0L && pair[1] > 0L) {
+ observedSum += pair[0];
+ cachedSum += pair[1];
+ }
+ }
+ if (cachedSum <= 0L || observedSum <= 0L) {
+ return 1.0;
+ }
+ double scale = (double) observedSum / (double) cachedSum;
+ if (scale < MIN_SCALE) {
+ return MIN_SCALE;
+ }
+ if (scale > MAX_SCALE) {
+ return MAX_SCALE;
+ }
+ return scale;
+ }
+
+ private static double fallbackAverageMs(Collection<Long>
completedDurations,
+ Map<String, Long> cachedTimings,
+ long elapsedTimeMs) {
+ if (completedDurations != null && !completedDurations.isEmpty()) {
+ long sum = 0L;
+ for (Long d : completedDurations) {
+ if (d != null && d > 0L) {
+ sum += d;
+ }
+ }
+ return sum / (double) completedDurations.size();
+ }
+ if (cachedTimings != null && !cachedTimings.isEmpty()) {
+ long sum = 0L;
+ for (Long d : cachedTimings.values()) {
+ if (d != null && d > 0L) {
+ sum += d;
+ }
+ }
+ return sum / (double) cachedTimings.size();
+ }
+ // Cold start: tiny placeholder so ETA is non-zero until the first
test finishes
+ return elapsedTimeMs > 0L ? elapsedTimeMs : 30_000L;
+ }
+
+ static Path cacheFile(String persistenceProvider) {
+ String normalized = (persistenceProvider == null ||
persistenceProvider.isEmpty())
? "unknown"
- : searchEngine.toLowerCase(Locale.ROOT);
- return Paths.get(System.getProperty("user.dir", "."),
".test-timing-cache-" + normalizedEngine + ".properties");
+ : persistenceProvider.toLowerCase(Locale.ROOT);
+ return Paths.get(System.getProperty("user.dir", "."),
".test-timing-cache-" + normalized + ".properties");
}
}
diff --git
a/itests/src/test/java/org/apache/unomi/itests/TestTimingCacheTest.java
b/itests/src/test/java/org/apache/unomi/itests/TestTimingCacheTest.java
new file mode 100644
index 000000000..a5030fe19
--- /dev/null
+++ b/itests/src/test/java/org/apache/unomi/itests/TestTimingCacheTest.java
@@ -0,0 +1,123 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.unomi.itests;
+
+import org.junit.After;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Unit tests for {@link TestTimingCache} ETA helpers and per-provider
isolation.
+ */
+public class TestTimingCacheTest {
+
+ private String previousUserDir;
+ private Path tempDir;
+
+ @Before
+ public void setUp() throws Exception {
+ previousUserDir = System.getProperty("user.dir");
+ tempDir = Files.createTempDirectory("unomi-test-timing-cache");
+ System.setProperty("user.dir", tempDir.toAbsolutePath().toString());
+ }
+
+ @After
+ public void tearDown() {
+ if (previousUserDir != null) {
+ System.setProperty("user.dir", previousUserDir);
+ }
+ }
+
+ @Test
+ public void cacheFilesAreIsolatedPerProvider() {
+ Path es = TestTimingCache.cacheFile("elasticsearch");
+ Path pg = TestTimingCache.cacheFile("postgresql");
+ Assert.assertNotEquals(es, pg);
+
Assert.assertTrue(es.getFileName().toString().contains("elasticsearch"));
+ Assert.assertTrue(pg.getFileName().toString().contains("postgresql"));
+ }
+
+ @Test
+ public void saveAndLoadRoundTripPerProvider() {
+ Map<String, Long> esTimings = Collections.singletonMap("FooIT#bar",
1_000L);
+ Map<String, Long> pgTimings = Collections.singletonMap("FooIT#bar",
5_000L);
+
+ TestTimingCache.save("elasticsearch", esTimings);
+ TestTimingCache.save("postgresql", pgTimings);
+
+ Assert.assertEquals(Long.valueOf(1_000L),
TestTimingCache.load("elasticsearch").get("FooIT#bar"));
+ Assert.assertEquals(Long.valueOf(5_000L),
TestTimingCache.load("postgresql").get("FooIT#bar"));
+ Assert.assertTrue(TestTimingCache.load("opensearch").isEmpty());
+ }
+
+ @Test
+ public void computeScaleDefaultsToOneWithoutPairs() {
+ Assert.assertEquals(1.0,
TestTimingCache.computeScale(Collections.emptyList()), 0.0);
+ Assert.assertEquals(1.0, TestTimingCache.computeScale(null), 0.0);
+ }
+
+ @Test
+ public void computeScaleUsesObservedOverCachedRatioAndClamps() {
+ List<long[]> slower = Collections.singletonList(new long[]{2_000L,
1_000L});
+ Assert.assertEquals(2.0, TestTimingCache.computeScale(slower), 0.0);
+
+ List<long[]> tooFast = Collections.singletonList(new long[]{10L,
10_000L});
+ Assert.assertEquals(TestTimingCache.MIN_SCALE,
TestTimingCache.computeScale(tooFast), 0.0);
+
+ List<long[]> tooSlow = Collections.singletonList(new long[]{50_000L,
1_000L});
+ Assert.assertEquals(TestTimingCache.MAX_SCALE,
TestTimingCache.computeScale(tooSlow), 0.0);
+ }
+
+ @Test
+ public void estimateRemainingUsesScaledHistoryAndFallbackAverage() {
+ Map<String, Long> cached = new HashMap<>();
+ cached.put("A#a", 1_000L);
+ cached.put("B#b", 2_000L);
+
+ Set<String> remaining = new HashSet<>(Arrays.asList("A#a", "C#c"));
+ List<long[]> observedVsCached = Collections.singletonList(new
long[]{1_500L, 1_000L});
+ List<Long> completed = Collections.singletonList(1_500L);
+
+ // scale = 1.5 → A contributes 1500; C uncached → fallback avg 1500
+ long eta = TestTimingCache.estimateRemainingMs(remaining, cached,
observedVsCached, completed, 0L);
+ Assert.assertEquals(3_000L, eta);
+ }
+
+ @Test
+ public void estimateRemainingUsesHistoricalAverageWhenNothingCompleted() {
+ Map<String, Long> cached = new HashMap<>();
+ cached.put("A#a", 1_000L);
+ cached.put("B#b", 3_000L);
+
+ Set<String> remaining = Collections.singleton("C#c");
+ long eta = TestTimingCache.estimateRemainingMs(
+ remaining, cached, Collections.emptyList(),
Collections.emptyList(), 0L);
+ // avg of history = 2000
+ Assert.assertEquals(2_000L, eta);
+ }
+}
diff --git
a/itests/src/test/java/org/apache/unomi/itests/migration/Migrate16xToCurrentVersionIT.java
b/itests/src/test/java/org/apache/unomi/itests/migration/Migrate16xToCurrentVersionIT.java
index 214f71fa8..bd03f8436 100644
---
a/itests/src/test/java/org/apache/unomi/itests/migration/Migrate16xToCurrentVersionIT.java
+++
b/itests/src/test/java/org/apache/unomi/itests/migration/Migrate16xToCurrentVersionIT.java
@@ -26,19 +26,23 @@ import org.apache.unomi.api.conditions.ConditionType;
import org.apache.unomi.api.tenants.Tenant;
import org.apache.unomi.geonames.services.GeonameEntry;
import org.apache.unomi.itests.BaseIT;
+import org.apache.unomi.itests.persistence.SearchBackendIT;
import org.apache.unomi.persistence.spi.aggregate.TermsAggregate;
import org.apache.unomi.shell.migration.utils.HttpUtils;
import org.apache.unomi.shell.migration.utils.MigrationUtils;
import org.junit.After;
import org.junit.Assert;
+import org.junit.Assume;
import org.junit.Before;
import org.junit.Test;
+import org.junit.experimental.categories.Category;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
import java.util.*;
+@Category(SearchBackendIT.class)
public class Migrate16xToCurrentVersionIT extends BaseIT {
private final static Logger LOGGER =
LoggerFactory.getLogger(Migrate16xToCurrentVersionIT.class);
@@ -54,7 +58,7 @@ public class Migrate16xToCurrentVersionIT extends BaseIT {
// Elasticsearch connection constants
private static String getEsBaseUrl() {
- return "http://localhost:"; + getSearchPort();
+ return getSearchEngineBaseUrl();
}
private static String getEsSnapshotRepo() {
return getEsBaseUrl() + "/_snapshot/snapshots_repository/";
@@ -109,8 +113,9 @@ public class Migrate16xToCurrentVersionIT extends BaseIT {
// This is called from BaseIT and will run before any migration setup
checkSearchEngine();
- if (SEARCH_ENGINE_OPENSEARCH.equals(searchEngine)) {
- System.out.println("Migration from 1.x to 2.x not supported for
OpenSearch, skipping snapshot restore");
+ if (!persistenceCapabilities().snapshotRestoreMigration()) {
+ System.out.println("snapshotRestoreMigration not supported for
provider "
+ + getPersistenceBackend().providerId() + " — starting
Unomi without legacy snapshot restore");
super.waitForStartup();
return;
}
@@ -119,7 +124,7 @@ public class Migrate16xToCurrentVersionIT extends BaseIT {
LOGGER.info("Restoring snapshot into search engine...");
// Restore snapshot from 1.6.x
- try (CloseableHttpClient httpClient = HttpUtils.initHttpClient(true,
null)) {
+ try (CloseableHttpClient httpClient = createSearchEngineHttpClient()) {
// Create snapshot repo
HttpUtils.executePutRequest(httpClient, getEsSnapshotRepo(),
resourceAsString(RESOURCE_CREATE_SNAPSHOTS_REPO), null);
// Get snapshot, insure it exists
@@ -199,10 +204,10 @@ public class Migrate16xToCurrentVersionIT extends BaseIT {
*/
@Test
public void checkMigratedData() throws Exception {
- if (SEARCH_ENGINE_OPENSEARCH.equals(searchEngine)) {
- System.out.println("Migration from 1.x to 2.x not supported for
OpenSearch, skipping checks");
- return;
- }
+ Assume.assumeTrue(
+ "snapshotRestoreMigration not supported for provider "
+ + getPersistenceBackend().providerId(),
+ persistenceCapabilities().snapshotRestoreMigration());
checkMergedProfilesAliases();
checkProfileInterests();
checkProfileTotalNbOfVisits();
@@ -654,10 +659,10 @@ public class Migrate16xToCurrentVersionIT extends BaseIT {
* Test that the default tenant was created during migration
(migrate-3.1.0-10-tenantInitialization)
*/
private void checkDefaultTenantCreated() throws Exception {
- if (SEARCH_ENGINE_OPENSEARCH.equals(searchEngine)) {
- System.out.println("Migration from 1.x to 2.x not supported for
OpenSearch, skipping checks");
- return;
- }
+ Assume.assumeTrue(
+ "snapshotRestoreMigration not supported for provider "
+ + getPersistenceBackend().providerId(),
+ persistenceCapabilities().snapshotRestoreMigration());
// Check that the default tenant index exists
Assert.assertTrue("Default tenant index should exist",
MigrationUtils.indexExists(httpClient, getEsBaseUrl(), INDEX_PREFIX_CONTEXT +
"tenant"));
@@ -692,10 +697,10 @@ public class Migrate16xToCurrentVersionIT extends BaseIT {
* have proper tenant information, audit metadata, and are accessible via
definitionsService.
*/
private void checkDefinitionsServiceObjectsAccessible() throws Exception {
- if (SEARCH_ENGINE_OPENSEARCH.equals(searchEngine)) {
- System.out.println("Migration from 1.x to 2.x not supported for
OpenSearch, skipping checks");
- return;
- }
+ Assume.assumeTrue(
+ "snapshotRestoreMigration not supported for provider "
+ + getPersistenceBackend().providerId(),
+ persistenceCapabilities().snapshotRestoreMigration());
// Refresh the definitions service cache to ensure migrated items are
loaded
// This is necessary because items might be in persistence but not yet
in cache
@@ -811,10 +816,10 @@ public class Migrate16xToCurrentVersionIT extends BaseIT {
* all condition types that use legacy *ESQueryBuilder syntax to use the
new generic QueryBuilder syntax.
*/
private void checkLegacyQueryBuilderMigration() throws Exception {
- if (SEARCH_ENGINE_OPENSEARCH.equals(searchEngine)) {
- System.out.println("Migration from 1.x to 2.x not supported for
OpenSearch, skipping checks");
- return;
- }
+ Assume.assumeTrue(
+ "snapshotRestoreMigration not supported for provider "
+ + getPersistenceBackend().providerId(),
+ persistenceCapabilities().snapshotRestoreMigration());
// Refresh the definitions service cache to ensure migrated items are
loaded
definitionsService.refresh();
diff --git
a/itests/src/test/java/org/apache/unomi/itests/persistence/ElasticsearchITBackend.java
b/itests/src/test/java/org/apache/unomi/itests/persistence/ElasticsearchITBackend.java
new file mode 100644
index 000000000..467c9d0fd
--- /dev/null
+++
b/itests/src/test/java/org/apache/unomi/itests/persistence/ElasticsearchITBackend.java
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License
+ */
+package org.apache.unomi.itests.persistence;
+
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.unomi.shell.migration.utils.HttpUtils;
+import org.ops4j.pax.exam.Option;
+
+import java.io.IOException;
+
+import static org.ops4j.pax.exam.CoreOptions.maven;
+import static
org.ops4j.pax.exam.karaf.options.KarafDistributionOption.editConfigurationFilePut;
+import static
org.ops4j.pax.exam.karaf.options.KarafDistributionOption.features;
+
+/**
+ * Built-in Elasticsearch backend for Unomi integration tests.
+ */
+public class ElasticsearchITBackend implements PersistenceITBackend {
+
+ @Override
+ public String providerId() {
+ return PersistenceITBackendResolver.PROVIDER_ELASTICSEARCH;
+ }
+
+ @Override
+ public Option[] featureOptions() {
+ return new Option[]{
+ features(
+
maven().groupId("org.apache.unomi").artifactId("unomi-kar").versionAsInProject().type("xml").classifier("features"),
+ "unomi-base",
+ "unomi-startup",
+ "unomi-elasticsearch-core",
+ "unomi-persistence-core",
+ "unomi-services",
+ "unomi-cxs-privacy-extension-services",
+ "unomi-plugins-base",
+ "unomi-plugins-request",
+ "unomi-plugins-mail",
+ "unomi-plugins-optimization-test",
+ "unomi-rest-api",
+ "unomi-cxs-privacy-extension",
+ "unomi-elasticsearch-conditions",
+ "unomi-cxs-lists-extension",
+ "unomi-cxs-geonames-extension",
+ "unomi-shell-dev-commands",
+ "unomi-wab",
+ "unomi-web-tracker",
+ "unomi-healthcheck-elasticsearch",
+ "unomi-router-karaf-feature",
+ "unomi-groovy-actions",
+ "unomi-rest-ui",
+ "cdp-graphql-feature",
+ "unomi-startup-complete"
+ ),
+ features(
+
maven().groupId("org.apache.unomi").artifactId("unomi-distribution").versionAsInProject().type("xml").classifier("features"),
+ "unomi-distribution-elasticsearch-graphql"
+ )
+ };
+ }
+
+ @Override
+ public Option[] configurationOptions() {
+ String port = searchPort();
+ return new Option[]{
+ editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.elasticsearch.cluster.name", "contextElasticSearchITests"),
+ editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.elasticsearch.addresses", "localhost:" + port),
+ editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.elasticsearch.taskWaitingPollingInterval", "50"),
+ editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.elasticsearch.rollover.maxDocs", "300"),
+ editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.elasticsearch.minimalClusterState", "YELLOW"),
+ };
+ }
+
+ @Override
+ public String distributionFeature() {
+ return "unomi-distribution-elasticsearch";
+ }
+
+ @Override
+ public String persistenceConfigPid() {
+ return "org.apache.unomi.persistence.elasticsearch";
+ }
+
+ @Override
+ public PersistenceITCapabilities capabilities() {
+ return PersistenceITCapabilities.elasticsearch();
+ }
+
+ @Override
+ public String searchBaseUrl() {
+ return "http://localhost:"; + searchPort();
+ }
+
+ @Override
+ public String searchPort() {
+ return System.getProperty("elasticsearch.port", "9400");
+ }
+
+ @Override
+ public CloseableHttpClient createSearchHttpClient() throws IOException {
+ return HttpUtils.initHttpClient(true, null);
+ }
+}
diff --git
a/itests/src/test/java/org/apache/unomi/itests/persistence/OpenSearchITBackend.java
b/itests/src/test/java/org/apache/unomi/itests/persistence/OpenSearchITBackend.java
new file mode 100644
index 000000000..04e182264
--- /dev/null
+++
b/itests/src/test/java/org/apache/unomi/itests/persistence/OpenSearchITBackend.java
@@ -0,0 +1,130 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License
+ */
+package org.apache.unomi.itests.persistence;
+
+import org.apache.http.auth.AuthScope;
+import org.apache.http.auth.UsernamePasswordCredentials;
+import org.apache.http.impl.client.BasicCredentialsProvider;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.unomi.shell.migration.utils.HttpUtils;
+import org.ops4j.pax.exam.Option;
+
+import java.io.IOException;
+
+import static org.ops4j.pax.exam.CoreOptions.maven;
+import static
org.ops4j.pax.exam.karaf.options.KarafDistributionOption.editConfigurationFilePut;
+import static
org.ops4j.pax.exam.karaf.options.KarafDistributionOption.features;
+
+/**
+ * Built-in OpenSearch backend for Unomi integration tests.
+ */
+public class OpenSearchITBackend implements PersistenceITBackend {
+
+ private static final String OS_USER = "admin";
+ private static final String OS_PASSWORD = "Unomi.1ntegrat10n.Tests";
+
+ @Override
+ public String providerId() {
+ return PersistenceITBackendResolver.PROVIDER_OPENSEARCH;
+ }
+
+ @Override
+ public Option[] featureOptions() {
+ return new Option[]{
+ features(
+
maven().groupId("org.apache.unomi").artifactId("unomi-kar").versionAsInProject().type("xml").classifier("features"),
+ "unomi-base",
+ "unomi-startup",
+ "unomi-opensearch-core",
+ "unomi-persistence-core",
+ "unomi-services",
+ "unomi-cxs-privacy-extension-services",
+ "unomi-plugins-base",
+ "unomi-plugins-request",
+ "unomi-plugins-mail",
+ "unomi-plugins-optimization-test",
+ "unomi-rest-api",
+ "unomi-cxs-privacy-extension",
+ "unomi-opensearch-conditions",
+ "unomi-cxs-lists-extension",
+ "unomi-cxs-geonames-extension",
+ "unomi-shell-dev-commands",
+ "unomi-wab",
+ "unomi-web-tracker",
+ "unomi-healthcheck-opensearch",
+ "unomi-router-karaf-feature",
+ "unomi-groovy-actions",
+ "unomi-rest-ui",
+ "cdp-graphql-feature",
+ "unomi-startup-complete"
+ ),
+ features(
+
maven().groupId("org.apache.unomi").artifactId("unomi-distribution").versionAsInProject().type("xml").classifier("features"),
+ "unomi-distribution-opensearch-graphql"
+ )
+ };
+ }
+
+ @Override
+ public Option[] configurationOptions() {
+ String port = searchPort();
+ return new Option[]{
+ editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.opensearch.cluster.name", "contextElasticSearchITests"),
+ editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.opensearch.addresses", "localhost:" + port),
+ editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.opensearch.username", OS_USER),
+ editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.opensearch.password", OS_PASSWORD),
+ editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.opensearch.sslEnable", "false"),
+ editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.opensearch.sslTrustAllCertificates", "true"),
+ editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.opensearch.rollover.maxDocs", "300"),
+ editConfigurationFilePut("etc/custom.system.properties",
"org.apache.unomi.opensearch.minimalClusterState", "YELLOW"),
+ };
+ }
+
+ @Override
+ public String distributionFeature() {
+ return "unomi-distribution-opensearch";
+ }
+
+ @Override
+ public String persistenceConfigPid() {
+ return "org.apache.unomi.persistence.opensearch";
+ }
+
+ @Override
+ public PersistenceITCapabilities capabilities() {
+ return PersistenceITCapabilities.opensearch();
+ }
+
+ @Override
+ public String searchBaseUrl() {
+ return "http://localhost:"; + searchPort();
+ }
+
+ @Override
+ public String searchPort() {
+ return System.getProperty("org.apache.unomi.opensearch.addresses",
"localhost:9401")
+ .split(":")[1];
+ }
+
+ @Override
+ public CloseableHttpClient createSearchHttpClient() throws IOException {
+ BasicCredentialsProvider credentialsProvider = new
BasicCredentialsProvider();
+ credentialsProvider.setCredentials(AuthScope.ANY,
+ new UsernamePasswordCredentials(OS_USER, OS_PASSWORD));
+ return HttpUtils.initHttpClient(true, credentialsProvider);
+ }
+}
diff --git
a/itests/src/test/java/org/apache/unomi/itests/persistence/PersistenceITBackend.java
b/itests/src/test/java/org/apache/unomi/itests/persistence/PersistenceITBackend.java
new file mode 100644
index 000000000..50beaa657
--- /dev/null
+++
b/itests/src/test/java/org/apache/unomi/itests/persistence/PersistenceITBackend.java
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License
+ */
+package org.apache.unomi.itests.persistence;
+
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.osgi.framework.BundleContext;
+import org.osgi.service.cm.ConfigurationAdmin;
+import org.ops4j.pax.exam.Option;
+
+import java.io.IOException;
+
+/**
+ * Test-only contract for provisioning and awaiting a {@code
PersistenceService} provider
+ * during Pax Exam integration tests.
+ * <p>
+ * Built-in implementations cover Elasticsearch and OpenSearch. Additional
providers register
+ * via {@link java.util.ServiceLoader} or {@code
-Dunomi.persistence.it.backend=<fqcn>}.
+ */
+public interface PersistenceITBackend {
+
+ /** Provider id (e.g. {@code elasticsearch}, {@code opensearch}). */
+ String providerId();
+
+ /**
+ * Karaf feature options for this provider (kar features + distribution
feature).
+ */
+ Option[] featureOptions();
+
+ /**
+ * Provider-specific {@code etc/custom.system.properties} (and related)
Pax Exam options.
+ */
+ Option[] configurationOptions();
+
+ /** Distribution feature name passed to {@code unomi:setup -d=…}. */
+ String distributionFeature();
+
+ /**
+ * Wait until the backend is usable before Unomi starts (cluster health,
JDBC ping, …).
+ * Default is a no-op; search backends typically poll HTTP health from
{@code BaseIT}.
+ * <p>
+ * Called from the Pax Exam driver / early {@code @Before} path — do not
assume OSGi
+ * services are available yet. Prefer {@link #prepareBeforeUnomiSetup} for
ConfigAdmin work.
+ */
+ default void awaitBackendReady() throws Exception {
+ }
+
+ /**
+ * Called in the Karaf JVM after {@code UnomiManagementService} is
available and before
+ * {@code unomi:setup}. Use for ConfigAdmin DataSource patching, bundle
restarts, etc.
+ */
+ default void prepareBeforeUnomiSetup(BundleContext bundleContext,
ConfigurationAdmin configurationAdmin)
+ throws Exception {
+ }
+
+ /** Optional verification after Unomi has started. */
+ default void assertHealthyAfterUnomiStart() throws Exception {
+ }
+
+ /** ConfigAdmin PID for {@code org.apache.unomi.persistence.*} knobs (e.g.
{@code throwExceptions}). */
+ String persistenceConfigPid();
+
+ PersistenceITCapabilities capabilities();
+
+ /** HTTP base URL when {@link PersistenceITCapabilities#httpAdminApi()} is
true. */
+ default String searchBaseUrl() {
+ throw new UnsupportedOperationException(providerId() + " does not
expose an HTTP admin API");
+ }
+
+ /** Search HTTP listen port when {@link
PersistenceITCapabilities#httpAdminApi()} is true. */
+ default String searchPort() {
+ throw new UnsupportedOperationException(providerId() + " does not
expose an HTTP admin API");
+ }
+
+ /** HTTP client for admin APIs (may include basic auth). */
+ default CloseableHttpClient createSearchHttpClient() throws IOException {
+ throw new UnsupportedOperationException(providerId() + " does not
expose an HTTP admin API");
+ }
+}
diff --git
a/itests/src/test/java/org/apache/unomi/itests/persistence/PersistenceITBackendResolver.java
b/itests/src/test/java/org/apache/unomi/itests/persistence/PersistenceITBackendResolver.java
new file mode 100644
index 000000000..5c8466237
--- /dev/null
+++
b/itests/src/test/java/org/apache/unomi/itests/persistence/PersistenceITBackendResolver.java
@@ -0,0 +1,110 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License
+ */
+package org.apache.unomi.itests.persistence;
+
+import java.util.ArrayList;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.ServiceLoader;
+
+/**
+ * Resolves the active {@link PersistenceITBackend} for the IT harness.
+ * <p>
+ * Resolution order:
+ * <ol>
+ * <li>{@code
-Dunomi.persistence.it.backend=<fully.qualified.ClassName>}</li>
+ * <li>{@link ServiceLoader} implementations matching the provider id</li>
+ * <li>Built-in Elasticsearch / OpenSearch backends</li>
+ * </ol>
+ * Provider id comes from {@code unomi.persistence.provider}, falling back to
the
+ * deprecated {@code unomi.search.engine} alias (default {@code
elasticsearch}).
+ */
+public final class PersistenceITBackendResolver {
+
+ public static final String PROVIDER_PROPERTY =
"unomi.persistence.provider";
+ /** @deprecated use {@link #PROVIDER_PROPERTY}; kept for existing scripts
and CI. */
+ @Deprecated
+ public static final String SEARCH_ENGINE_PROPERTY = "unomi.search.engine";
+ public static final String BACKEND_CLASS_PROPERTY =
"unomi.persistence.it.backend";
+
+ public static final String PROVIDER_ELASTICSEARCH = "elasticsearch";
+ public static final String PROVIDER_OPENSEARCH = "opensearch";
+
+ private PersistenceITBackendResolver() {
+ }
+
+ public static String resolveProviderId() {
+ String provider = System.getProperty(PROVIDER_PROPERTY);
+ if (provider != null && !provider.isBlank()) {
+ return provider.trim();
+ }
+ return System.getProperty(SEARCH_ENGINE_PROPERTY,
PROVIDER_ELASTICSEARCH).trim();
+ }
+
+ public static PersistenceITBackend resolve() {
+ String explicitClass = System.getProperty(BACKEND_CLASS_PROPERTY);
+ if (explicitClass != null && !explicitClass.isBlank()) {
+ return instantiate(explicitClass.trim());
+ }
+
+ String providerId = resolveProviderId();
+ Map<String, PersistenceITBackend> byId = loadBackendsById();
+ PersistenceITBackend backend = byId.get(providerId);
+ if (backend != null) {
+ return backend;
+ }
+
+ throw new IllegalArgumentException(
+ "No PersistenceITBackend registered for provider '" +
providerId + "'. "
+ + "Set -D" + PROVIDER_PROPERTY + "=<id> (or deprecated
-D" + SEARCH_ENGINE_PROPERTY + "), "
+ + "register a ServiceLoader implementation, or set -D"
+ BACKEND_CLASS_PROPERTY + "=<fqcn>. "
+ + "Known built-in / discovered ids: " + byId.keySet());
+ }
+
+ private static Map<String, PersistenceITBackend> loadBackendsById() {
+ Map<String, PersistenceITBackend> byId = new LinkedHashMap<>();
+ // Built-ins first so ServiceLoader can override the same id if
desired.
+ register(byId, new ElasticsearchITBackend());
+ register(byId, new OpenSearchITBackend());
+ for (PersistenceITBackend backend :
ServiceLoader.load(PersistenceITBackend.class)) {
+ register(byId, backend);
+ }
+ return byId;
+ }
+
+ private static void register(Map<String, PersistenceITBackend> byId,
PersistenceITBackend backend) {
+ byId.put(backend.providerId(), backend);
+ }
+
+ private static PersistenceITBackend instantiate(String fqcn) {
+ try {
+ Class<?> clazz = Class.forName(fqcn);
+ if (!PersistenceITBackend.class.isAssignableFrom(clazz)) {
+ throw new IllegalArgumentException(fqcn + " does not implement
PersistenceITBackend");
+ }
+ return (PersistenceITBackend)
clazz.getDeclaredConstructor().newInstance();
+ } catch (ReflectiveOperationException e) {
+ throw new IllegalArgumentException("Failed to instantiate
PersistenceITBackend " + fqcn, e);
+ }
+ }
+
+ /** Visible for tests / diagnostics. */
+ static List<String> knownProviderIds() {
+ return new ArrayList<>(loadBackendsById().keySet());
+ }
+}
diff --git
a/itests/src/test/java/org/apache/unomi/itests/persistence/PersistenceITCapabilities.java
b/itests/src/test/java/org/apache/unomi/itests/persistence/PersistenceITCapabilities.java
new file mode 100644
index 000000000..707bc4301
--- /dev/null
+++
b/itests/src/test/java/org/apache/unomi/itests/persistence/PersistenceITCapabilities.java
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License
+ */
+package org.apache.unomi.itests.persistence;
+
+/**
+ * Backend-agnostic IT capabilities. Describe <em>what</em> the harness and
tests may
+ * rely on (HTTP admin surface, rollover API shape, health probes), not which
vendor
+ * product is installed.
+ * <p>
+ * Built-in Elasticsearch / OpenSearch factories set these to match today’s
search
+ * engines; JDBC and other providers typically use {@link #none()}.
+ */
+public final class PersistenceITCapabilities {
+
+ /**
+ * HTTP API shape used to inspect or assert index rollover policy wiring.
+ * Named after the protocol, not a vendor.
+ */
+ public enum IndexRolloverApi {
+ /** No rollover HTTP API (typical for SQL / embedded stores). */
+ NONE,
+ /**
+ * Lifecycle-policy HTTP API (paths/settings of the form {@code _ilm},
+ * {@code index.lifecycle.*}).
+ */
+ LIFECYCLE,
+ /**
+ * State-management-policy HTTP API (paths/settings of the form
+ * {@code _plugins/_ism}, {@code index_state_management.*}).
+ */
+ STATE_MANAGEMENT;
+
+ public boolean isPresent() {
+ return this != NONE;
+ }
+ }
+
+ /**
+ * Expected outcome of range queries on flattened / nested document
properties.
+ */
+ public enum FlattenedRangeQueryResult {
+ /** Query succeeds with null or an empty hit list. */
+ EMPTY,
+ /** Query may return matching documents. */
+ HITS
+ }
+
+ private final boolean httpAdminApi;
+ private final boolean providerNamedHealthProbe;
+ private final boolean clusterHealthProbe;
+ private final IndexRolloverApi indexRolloverApi;
+ private final boolean snapshotRestoreMigration;
+ private final FlattenedRangeQueryResult flattenedRangeQueryResult;
+
+ private PersistenceITCapabilities(Builder builder) {
+ this.httpAdminApi = builder.httpAdminApi;
+ this.providerNamedHealthProbe = builder.providerNamedHealthProbe;
+ this.clusterHealthProbe = builder.clusterHealthProbe;
+ this.indexRolloverApi = builder.indexRolloverApi;
+ this.snapshotRestoreMigration = builder.snapshotRestoreMigration;
+ this.flattenedRangeQueryResult = builder.flattenedRangeQueryResult;
+ }
+
+ /**
+ * Backend exposes HTTP admin APIs for cluster/index/snapshot operations
+ * (typical of search engines). When false, {@code BaseIT} skips HTTP
health
+ * prep and search helpers refuse to run.
+ */
+ public boolean httpAdminApi() {
+ return httpAdminApi;
+ }
+
+ /**
+ * Health check includes a probe whose name equals the persistence
provider id.
+ */
+ public boolean providerNamedHealthProbe() {
+ return providerNamedHealthProbe;
+ }
+
+ /**
+ * Health check includes a cluster-level probe (name {@code cluster}).
+ */
+ public boolean clusterHealthProbe() {
+ return clusterHealthProbe;
+ }
+
+ /**
+ * Rollover policy HTTP API shape, or {@link IndexRolloverApi#NONE}.
+ */
+ public IndexRolloverApi indexRolloverApi() {
+ return indexRolloverApi;
+ }
+
+ /**
+ * Backend can exercise legacy snapshot-restore migration fixtures
+ * (search-engine snapshot HTTP + restore into current indices).
+ */
+ public boolean snapshotRestoreMigration() {
+ return snapshotRestoreMigration;
+ }
+
+ /**
+ * How range queries on flattened properties behave for assertions.
+ */
+ public FlattenedRangeQueryResult flattenedRangeQueryResult() {
+ return flattenedRangeQueryResult;
+ }
+
+ public static Builder builder() {
+ return new Builder();
+ }
+
+ /** No optional capabilities — JDBC, embedded, or incomplete providers. */
+ public static PersistenceITCapabilities none() {
+ return builder().build();
+ }
+
+ /** @deprecated use {@link #none()} */
+ @Deprecated
+ public static PersistenceITCapabilities nonSearchBackend() {
+ return none();
+ }
+
+ public static PersistenceITCapabilities elasticsearch() {
+ return builder()
+ .httpAdminApi(true)
+ .providerNamedHealthProbe(true)
+ .clusterHealthProbe(true)
+ .indexRolloverApi(IndexRolloverApi.LIFECYCLE)
+ .snapshotRestoreMigration(true)
+ .flattenedRangeQueryResult(FlattenedRangeQueryResult.EMPTY)
+ .build();
+ }
+
+ public static PersistenceITCapabilities opensearch() {
+ return builder()
+ .httpAdminApi(true)
+ .providerNamedHealthProbe(true)
+ .clusterHealthProbe(true)
+ .indexRolloverApi(IndexRolloverApi.STATE_MANAGEMENT)
+ .snapshotRestoreMigration(false)
+ .flattenedRangeQueryResult(FlattenedRangeQueryResult.HITS)
+ .build();
+ }
+
+ public static final class Builder {
+ private boolean httpAdminApi;
+ private boolean providerNamedHealthProbe;
+ private boolean clusterHealthProbe;
+ private IndexRolloverApi indexRolloverApi = IndexRolloverApi.NONE;
+ private boolean snapshotRestoreMigration;
+ private FlattenedRangeQueryResult flattenedRangeQueryResult =
FlattenedRangeQueryResult.EMPTY;
+
+ public Builder httpAdminApi(boolean httpAdminApi) {
+ this.httpAdminApi = httpAdminApi;
+ return this;
+ }
+
+ public Builder providerNamedHealthProbe(boolean
providerNamedHealthProbe) {
+ this.providerNamedHealthProbe = providerNamedHealthProbe;
+ return this;
+ }
+
+ public Builder clusterHealthProbe(boolean clusterHealthProbe) {
+ this.clusterHealthProbe = clusterHealthProbe;
+ return this;
+ }
+
+ public Builder indexRolloverApi(IndexRolloverApi indexRolloverApi) {
+ this.indexRolloverApi = indexRolloverApi != null ?
indexRolloverApi : IndexRolloverApi.NONE;
+ return this;
+ }
+
+ public Builder snapshotRestoreMigration(boolean
snapshotRestoreMigration) {
+ this.snapshotRestoreMigration = snapshotRestoreMigration;
+ return this;
+ }
+
+ public Builder flattenedRangeQueryResult(FlattenedRangeQueryResult
flattenedRangeQueryResult) {
+ this.flattenedRangeQueryResult = flattenedRangeQueryResult != null
+ ? flattenedRangeQueryResult
+ : FlattenedRangeQueryResult.EMPTY;
+ return this;
+ }
+
+ public PersistenceITCapabilities build() {
+ return new PersistenceITCapabilities(this);
+ }
+ }
+}
diff --git
a/itests/src/test/java/org/apache/unomi/itests/persistence/SearchBackendIT.java
b/itests/src/test/java/org/apache/unomi/itests/persistence/SearchBackendIT.java
new file mode 100644
index 000000000..9f8f81820
--- /dev/null
+++
b/itests/src/test/java/org/apache/unomi/itests/persistence/SearchBackendIT.java
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License
+ */
+package org.apache.unomi.itests.persistence;
+
+/**
+ * Marker category for ITs whose <em>primary</em> assertions need HTTP admin
APIs
+ * (snapshot restore, index rollover policy HTTP). Prefer keeping these
classes in
+ * {@link org.apache.unomi.itests.CorePersistenceITs} / {@link
org.apache.unomi.itests.AllITs}
+ * and gating with {@link org.junit.Assume} on {@link
PersistenceITCapabilities} so unsupported
+ * backends skip rather than losing suite membership.
+ * <p>
+ * Optional Failsafe {@code excludedGroups} may still list this category for
specialized cells.
+ */
+public interface SearchBackendIT {
+}
diff --git
a/itests/src/test/resources/META-INF/services/org.apache.unomi.itests.persistence.PersistenceITBackend
b/itests/src/test/resources/META-INF/services/org.apache.unomi.itests.persistence.PersistenceITBackend
new file mode 100644
index 000000000..17e9a7ed1
--- /dev/null
+++
b/itests/src/test/resources/META-INF/services/org.apache.unomi.itests.persistence.PersistenceITBackend
@@ -0,0 +1,19 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+org.apache.unomi.itests.persistence.ElasticsearchITBackend
+org.apache.unomi.itests.persistence.OpenSearchITBackend
