This is an automated email from the ASF dual-hosted git repository. sergehuber pushed a commit to branch UNOMI-968-pluggable-persistence-itests in repository https://gitbox.apache.org/repos/asf/unomi.git
commit fcc8d2ef699b7a55340a66ce245bbcb9af52f4b8 Author: Serge Huber <[email protected]> AuthorDate: Mon Jul 20 23:25:44 2026 +0200 UNOMI-968: Pluggable persistence IT harness for ES, OS, and other backends Extract PersistenceITBackend SPI with capability-based skips so the same CorePersistenceITs/AllITs suite runs on Elasticsearch and OpenSearch as before, while external providers (PostgreSQL, etc.) can plug in via ServiceLoader without forking Unomi IT sources. --- itests/README.md | 84 +++++- itests/pom.xml | 32 +++ .../test/java/org/apache/unomi/itests/AllITs.java | 4 +- .../test/java/org/apache/unomi/itests/BaseIT.java | 309 +++++++++++---------- .../{AllITs.java => CorePersistenceITs.java} | 11 +- .../org/apache/unomi/itests/HealthCheckIT.java | 51 +++- .../java/org/apache/unomi/itests/JSONSchemaIT.java | 11 +- .../org/apache/unomi/itests/ProfileServiceIT.java | 6 +- .../itests/ProfileServiceWithoutOverwriteIT.java | 3 +- .../java/org/apache/unomi/itests/RolloverIT.java | 42 ++- .../migration/Migrate16xToCurrentVersionIT.java | 45 +-- .../itests/persistence/ElasticsearchITBackend.java | 117 ++++++++ .../itests/persistence/OpenSearchITBackend.java | 130 +++++++++ .../itests/persistence/PersistenceITBackend.java | 92 ++++++ .../persistence/PersistenceITBackendResolver.java | 110 ++++++++ .../persistence/PersistenceITCapabilities.java | 204 ++++++++++++++ .../unomi/itests/persistence/SearchBackendIT.java | 29 ++ ...e.unomi.itests.persistence.PersistenceITBackend | 2 + 18 files changed, 1076 insertions(+), 206 deletions(-) diff --git a/itests/README.md b/itests/README.md index ebb3d9708..20a5f79ed 100644 --- a/itests/README.md +++ b/itests/README.md @@ -32,7 +32,9 @@ and data migration. `docker-maven-plugin`). - **Maven Failsafe** runs a single entry point — `AllITs` — which aggregates all test classes. Each test class extends `BaseIT`, which handles Karaf startup, OSGi service - injection, and common test utilities. + injection, and common test utilities. Persistence is selected via + `unomi.persistence.provider` (Elasticsearch / OpenSearch today; see + [Pluggable persistence providers](#pluggable-persistence-providers)). A full IT run typically takes 20–30 minutes. The Karaf instance is created fresh for each run under `itests/target/exam/` with a UUID directory name. @@ -284,6 +286,86 @@ See the [Maven Failsafe plugin docs](https://maven.apache.org/surefire/maven-fai --- +## Pluggable persistence providers + +Unomi’s runtime persistence is an OSGi capability (`unomi.persistence;provider:=…`). +The IT harness follows the same idea: `BaseIT` resolves a test-only +`PersistenceITBackend` instead of hard-coding Elasticsearch / OpenSearch. + +### Provider selection + +| Property | Role | +|----------|------| +| `unomi.persistence.provider` | Preferred provider id (`elasticsearch`, `opensearch`, or an extension id) | +| `unomi.search.engine` | **Deprecated alias** — used when `unomi.persistence.provider` is unset | +| `unomi.persistence.it.backend` | Optional FQCN of a `PersistenceITBackend` implementation | + +Default CI cells set both `unomi.persistence.provider` and the deprecated +`unomi.search.engine` so existing `--use-opensearch` / scripts keep working. + +### Built-in backends + +| Id | Class | Suite for CI | +|----|-------|--------------| +| `elasticsearch` | `ElasticsearchITBackend` | `AllITs` (default) | +| `opensearch` | `OpenSearchITBackend` | `AllITs` with `-Duse.opensearch=true` | + +### Search-only vs core behavioural tests + +`AllITs` and `CorePersistenceITs` share the **same** test membership for maximum coverage. +Tests that need HTTP admin / snapshot / rollover APIs stay in the suite and use +`Assume` + `PersistenceITCapabilities` so unsupported backends **skip** (reported as +skipped, not green false-pass): + +| Class | Gate | +|-------|------| +| `Migrate16xToCurrentVersionIT` | `snapshotRestoreMigration()` | +| `RolloverIT` | `indexRolloverApi().isPresent()` + `httpAdminApi()`; switch on `LIFECYCLE` / `STATE_MANAGEMENT` | +| `HealthCheckIT` | Always asserts `karaf` / `unomi` / `persistence`; optional `providerNamedHealthProbe` / `clusterHealthProbe` | + +`HealthCheckIT` is not search-only: it runs on every provider. + +Elasticsearch / OpenSearch CI continues to use `AllITs`. Portable / non-search cells +(PostgreSQL, …) should run `CorePersistenceITs` (identical membership, clearer name). + +```bash +mvn clean install -P integration-tests -Dit.test=org.apache.unomi.itests.CorePersistenceITs +``` + +Capability flags gate remaining special-cases +(e.g. `snapshotRestoreMigration`, `flattenedRangeQueryResult`, `indexRolloverApi`) +so providers skip what they cannot support instead of checking product names. + +### Adding a third-party backend + +1. Implement `org.apache.unomi.itests.persistence.PersistenceITBackend` + (feature options, distribution name for `unomi:setup`, ConfigAdmin PID, + capabilities, await-ready / HTTP helpers as needed). +2. Put the implementation on the Failsafe test classpath. +3. Register it either: + - in `META-INF/services/org.apache.unomi.itests.persistence.PersistenceITBackend`, or + - via `-Dunomi.persistence.it.backend=com.example.MyPersistenceITBackend` +4. Select it with `-Dunomi.persistence.provider=<your-id>`. +5. Run `CorePersistenceITs` (same membership as `AllITs`). Incomplete SPI + implementations may still fail behavioural tests — unsupported HTTP-admin + features are skipped via `Assume`, not removed from the suite. + +`unomi-itests` publishes a **test-jar** (`mvn -pl itests install`) so external modules can +depend on `BaseIT`, `CorePersistenceITs`, and `PersistenceITBackend` without forking +sources. Optionally implement `prepareBeforeUnomiSetup` to patch ConfigAdmin (e.g. JDBC +DataSource) after `UnomiManagementService` is up and before `unomi:setup`. + +Example Failsafe exclude when running `AllITs` against a non-search provider: + +```xml +<excludedGroups>org.apache.unomi.itests.persistence.SearchBackendIT</excludedGroups> +``` + +(Note: JUnit 4 category filtering applies to classes Failsafe launches directly; +prefer `CorePersistenceITs` when using the suite entry point.) + +--- + ## Debugging Integration Tests ### Attaching a remote debugger to the test JVM diff --git a/itests/pom.xml b/itests/pom.xml index d28e1d2a6..026a1798f 100644 --- a/itests/pom.xml +++ b/itests/pom.xml @@ -28,6 +28,7 @@ <description>Apache Unomi Context Server integration tests</description> <properties> + <unomi.persistence.provider>elasticsearch</unomi.persistence.provider> <unomi.search.engine>elasticsearch</unomi.search.engine> <use.opensearch>false</use.opensearch> <docker.container.name>itests-opensearch</docker.container.name> @@ -277,6 +278,33 @@ </systemPropertyVariables> </configuration> </plugin> + <!-- Publish test classes so out-of-tree PersistenceService providers can implement + PersistenceITBackend and reuse CorePersistenceITs / BaseIT (UNOMI-968). --> + <plugin> + <groupId>org.apache.maven.plugins</groupId> + <artifactId>maven-jar-plugin</artifactId> + <executions> + <execution> + <id>test-jar</id> + <goals> + <goal>test-jar</goal> + </goals> + <configuration> + <!-- Include IT classes and test resources (fixtures used by BaseIT.config). --> + <includes> + <include>org/apache/unomi/itests/**</include> + <include>**/*.cfg</include> + <include>**/*.csv</include> + <include>**/*.json</include> + <include>**/*.groovy</include> + <include>migration/**</include> + <include>schemas/**</include> + <include>META-INF/**</include> + </includes> + </configuration> + </execution> + </executions> + </plugin> </plugins> </build> @@ -306,6 +334,8 @@ </includes> <systemPropertyVariables> <my.system.property>foo</my.system.property> + <unomi.persistence.provider>elasticsearch</unomi.persistence.provider> + <!-- Deprecated alias; keep for older scripts --> <unomi.search.engine>elasticsearch</unomi.search.engine> <elasticsearch.port>${elasticsearch.port}</elasticsearch.port> <it.karaf.heap>${karaf.heap}</it.karaf.heap> @@ -432,6 +462,8 @@ </includes> <systemPropertyVariables> <my.system.property>foo</my.system.property> + <unomi.persistence.provider>opensearch</unomi.persistence.provider> + <!-- Deprecated alias; keep for older scripts --> <unomi.search.engine>opensearch</unomi.search.engine> <org.apache.unomi.opensearch.addresses>localhost:${opensearch.port}</org.apache.unomi.opensearch.addresses> <org.ops4j.pax.logging.DefaultServiceLog.level>INFO</org.ops4j.pax.logging.DefaultServiceLog.level> diff --git a/itests/src/test/java/org/apache/unomi/itests/AllITs.java b/itests/src/test/java/org/apache/unomi/itests/AllITs.java index 8617b1dcb..41351e5b0 100644 --- a/itests/src/test/java/org/apache/unomi/itests/AllITs.java +++ b/itests/src/test/java/org/apache/unomi/itests/AllITs.java @@ -24,7 +24,9 @@ import org.junit.runner.RunWith; import org.junit.runners.Suite.SuiteClasses; /** - * Defines suite of test classes to run. + * Defines suite of test classes to run (Elasticsearch / OpenSearch CI default). + * Same membership as {@link CorePersistenceITs}; use capabilities + {@code Assume} for + * provider-specific skips rather than maintaining a smaller suite. * * @author Sergiy Shyrkov */ diff --git a/itests/src/test/java/org/apache/unomi/itests/BaseIT.java b/itests/src/test/java/org/apache/unomi/itests/BaseIT.java index 8587f4f8a..5ec6385ba 100644 --- a/itests/src/test/java/org/apache/unomi/itests/BaseIT.java +++ b/itests/src/test/java/org/apache/unomi/itests/BaseIT.java @@ -22,8 +22,6 @@ import org.apache.camel.CamelContext; import org.apache.camel.Route; import org.apache.camel.ServiceStatus; import org.apache.commons.io.IOUtils; -import org.apache.http.auth.AuthScope; -import org.apache.http.auth.UsernamePasswordCredentials; import org.apache.http.HttpEntity; import org.apache.http.client.config.RequestConfig; import org.apache.http.client.methods.*; @@ -54,6 +52,9 @@ import org.apache.unomi.api.tenants.Tenant; import org.apache.unomi.api.tenants.TenantService; import org.apache.unomi.api.utils.ConditionBuilder; import org.apache.unomi.groovy.actions.services.GroovyActionsService; +import org.apache.unomi.itests.persistence.PersistenceITBackend; +import org.apache.unomi.itests.persistence.PersistenceITBackendResolver; +import org.apache.unomi.itests.persistence.PersistenceITCapabilities; import org.apache.unomi.itests.tools.LogChecker; import org.apache.unomi.itests.tools.httpclient.HttpClientThatWaitsForUnomi; import org.apache.unomi.lifecycle.BundleWatcher; @@ -98,9 +99,11 @@ import javax.net.ssl.X509TrustManager; import java.io.File; import java.io.IOException; import java.io.InputStream; +import java.net.URL; import java.nio.file.Files; import java.nio.file.Path; import java.nio.file.Paths; +import java.nio.file.StandardCopyOption; import java.security.KeyManagementException; import java.security.NoSuchAlgorithmException; import java.security.SecureRandom; @@ -135,16 +138,23 @@ public abstract class BaseIT extends KarafTestSupport { protected static final int DEFAULT_TRYING_TRIES = 10; protected static final int DEFAULT_SHOULDBETRUE_TRIES = 5; - protected static final String SEARCH_ENGINE_PROPERTY = "unomi.search.engine"; + /** @deprecated use {@link PersistenceITBackendResolver#PROVIDER_PROPERTY}. */ + @Deprecated + protected static final String SEARCH_ENGINE_PROPERTY = PersistenceITBackendResolver.SEARCH_ENGINE_PROPERTY; protected static final String SEARCH_ENGINE_HTTPREQUEST_LOG_LEVEL = "unomi.search.engine.httprequest.log.level"; - protected static final String SEARCH_ENGINE_ELASTICSEARCH = "elasticsearch"; - protected static final String SEARCH_ENGINE_OPENSEARCH = "opensearch"; + protected static final String SEARCH_ENGINE_ELASTICSEARCH = PersistenceITBackendResolver.PROVIDER_ELASTICSEARCH; + protected static final String SEARCH_ENGINE_OPENSEARCH = PersistenceITBackendResolver.PROVIDER_OPENSEARCH; + protected static final String PERSISTENCE_PROVIDER_PROPERTY = PersistenceITBackendResolver.PROVIDER_PROPERTY; protected static final String RESOLVER_DEBUG_PROPERTY = "it.unomi.resolver.debug"; protected static final String ENABLE_LOG_CHECKING_PROPERTY = "it.unomi.log.checking.enabled"; protected static final String CAMEL_DEBUG_PROPERTY = "it.unomi.camel.debug"; protected static boolean unomiStarted = false; + /** + * Active provider id. Kept for existing tests; prefer {@link #getPersistenceBackend()}. + */ protected static String searchEngine = SEARCH_ENGINE_ELASTICSEARCH; + private static PersistenceITBackend persistenceBackend; private static boolean searchEngineConfiguredForTesting = false; private static boolean searchEngineHealthVerifiedAfterStartup = false; @@ -239,37 +249,71 @@ public abstract class BaseIT extends KarafTestSupport { } protected void checkSearchEngine() { - searchEngine = System.getProperty(SEARCH_ENGINE_PROPERTY, SEARCH_ENGINE_ELASTICSEARCH); + resolvePersistenceBackend(); configureSearchEngineForTesting(); } + /** + * Resolves (and caches) the active {@link PersistenceITBackend} from system properties / + * ServiceLoader. Also updates the legacy {@link #searchEngine} field. + */ + protected static PersistenceITBackend resolvePersistenceBackend() { + if (persistenceBackend == null) { + persistenceBackend = PersistenceITBackendResolver.resolve(); + searchEngine = persistenceBackend.providerId(); + } + return persistenceBackend; + } + + protected static PersistenceITBackend getPersistenceBackend() { + return resolvePersistenceBackend(); + } + + protected PersistenceITCapabilities persistenceCapabilities() { + return getPersistenceBackend().capabilities(); + } + + /** ConfigAdmin PID for the active persistence provider (e.g. throwExceptions). */ + protected String persistenceConfigPid() { + return getPersistenceBackend().persistenceConfigPid(); + } + @Before public void waitForStartup() throws InterruptedException { // disable retry retry = new KarafTestSupport.Retry(false); - // Check search engine and apply any necessary fixes (e.g., default_template deletion) + // Resolve provider and apply any necessary backend prep (e.g., zero-replica template) checkSearchEngine(); + try { + getPersistenceBackend().awaitBackendReady(); + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); + throw e; + } catch (Exception e) { + throw new IllegalStateException("Persistence backend awaitBackendReady failed", e); + } // Start Unomi if not already done if (!unomiStarted) { // We must check that the Unomi Management Service is up and running before launching the // command otherwise the start configuration will not be properly populated. waitForUnomiManagementService(); - if (SEARCH_ENGINE_ELASTICSEARCH.equals(searchEngine)) { - LOGGER.info("Starting Unomi with elasticsearch search engine..."); - System.out.println("==== Starting Unomi with elasticsearch search engine..."); - executeCommand("unomi:setup -d=unomi-distribution-elasticsearch -f=true"); - executeCommand("unomi:start"); - } else if (SEARCH_ENGINE_OPENSEARCH.equals(searchEngine)){ - LOGGER.info("Starting Unomi with opensearch search engine..."); - System.out.println("==== Starting Unomi with opensearch search engine..."); - executeCommand("unomi:setup -d=unomi-distribution-opensearch -f=true"); - executeCommand("unomi:start"); - } else { - LOGGER.error("Unknown search engine: " + searchEngine); - throw new InterruptedException("Unknown search engine: " + searchEngine); + PersistenceITBackend backend = getPersistenceBackend(); + try { + backend.prepareBeforeUnomiSetup(bundleContext, configurationAdmin); + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); + throw e; + } catch (Exception e) { + throw new IllegalStateException("Persistence backend prepareBeforeUnomiSetup failed", e); } + String distribution = backend.distributionFeature(); + LOGGER.info("Starting Unomi with persistence provider {} (distribution {})...", backend.providerId(), distribution); + System.out.println("==== Starting Unomi with persistence provider " + backend.providerId() + + " (distribution " + distribution + ")..."); + executeCommand("unomi:setup -d=" + distribution + " -f=true"); + executeCommand("unomi:start"); unomiStarted = true; } @@ -542,6 +586,43 @@ public abstract class BaseIT extends KarafTestSupport { Thread.sleep(1000); } + /** + * Resolves a fixture under {@code src/test/resources} for Pax Exam {@code replaceConfigurationFile}. + * Prefers the local filesystem (Unomi itests module); falls back to the classpath so out-of-tree + * consumers of the {@code unomi-itests} test-jar can reuse {@link #config()} without copying files. + */ + protected static File resolveTestResource(String pathUnderSrcTestResources) { + File local = new File(pathUnderSrcTestResources); + if (local.isFile()) { + return local; + } + String classpathName = pathUnderSrcTestResources; + final String prefix = "src/test/resources/"; + if (classpathName.startsWith(prefix)) { + classpathName = classpathName.substring(prefix.length()); + } + URL url = BaseIT.class.getClassLoader().getResource(classpathName); + if (url == null) { + throw new IllegalStateException("Missing test resource: " + pathUnderSrcTestResources + + " (not on filesystem or classpath)"); + } + try { + if ("file".equals(url.getProtocol())) { + return new File(url.toURI()); + } + String fileName = new File(classpathName).getName(); + Path tmp = Files.createTempFile("unomi-it-resource-", "-" + fileName); + try (InputStream in = url.openStream()) { + Files.copy(in, tmp, StandardCopyOption.REPLACE_EXISTING); + } + File out = tmp.toFile(); + out.deleteOnExit(); + return out; + } catch (Exception e) { + throw new IllegalStateException("Cannot resolve test resource " + pathUnderSrcTestResources, e); + } + } + @Override public MavenArtifactUrlReference getKarafDistribution() { return maven().groupId("org.apache.unomi").artifactId("unomi").versionAsInProject().type("tar.gz"); @@ -554,120 +635,41 @@ public abstract class BaseIT extends KarafTestSupport { System.out.println("==== Configuring container"); } - searchEngine = System.getProperty(SEARCH_ENGINE_PROPERTY, SEARCH_ENGINE_ELASTICSEARCH); + resolvePersistenceBackend(); if (!configLogged) { - LOGGER.info("Search Engine: {}", searchEngine); - System.out.println("Search Engine: " + searchEngine); - } - - // Define features option based on search engine - Option featuresOption; - Option distributionOption; - if (SEARCH_ENGINE_ELASTICSEARCH.equals(searchEngine)) { - featuresOption = features( - maven().groupId("org.apache.unomi").artifactId("unomi-kar").versionAsInProject().type("xml").classifier("features"), - "unomi-base", - "unomi-startup", - "unomi-elasticsearch-core", - "unomi-persistence-core", - "unomi-services", - "unomi-cxs-privacy-extension-services", - "unomi-plugins-base", - "unomi-plugins-request", - "unomi-plugins-mail", - "unomi-plugins-optimization-test", - "unomi-rest-api", - "unomi-cxs-privacy-extension", - "unomi-elasticsearch-conditions", - "unomi-cxs-lists-extension", - "unomi-cxs-geonames-extension", - "unomi-shell-dev-commands", - "unomi-wab", - "unomi-web-tracker", - "unomi-healthcheck-elasticsearch", - "unomi-router-karaf-feature", - "unomi-groovy-actions", - "unomi-rest-ui", - "cdp-graphql-feature", - "unomi-startup-complete" - ); - distributionOption = features( - maven().groupId("org.apache.unomi").artifactId("unomi-distribution").versionAsInProject().type("xml").classifier("features"), - "unomi-distribution-elasticsearch-graphql" - ); - } else if (SEARCH_ENGINE_OPENSEARCH.equals(searchEngine)) { - featuresOption = features( - maven().groupId("org.apache.unomi").artifactId("unomi-kar").versionAsInProject().type("xml").classifier("features"), - "unomi-base", - "unomi-startup", - "unomi-opensearch-core", - "unomi-persistence-core", - "unomi-services", - "unomi-cxs-privacy-extension-services", - "unomi-plugins-base", - "unomi-plugins-request", - "unomi-plugins-mail", - "unomi-plugins-optimization-test", - "unomi-rest-api", - "unomi-cxs-privacy-extension", - "unomi-opensearch-conditions", - "unomi-cxs-lists-extension", - "unomi-cxs-geonames-extension", - "unomi-shell-dev-commands", - "unomi-wab", - "unomi-web-tracker", - "unomi-healthcheck-opensearch", - "unomi-router-karaf-feature", - "unomi-groovy-actions", - "unomi-rest-ui", - "cdp-graphql-feature", - "unomi-startup-complete" - ); - distributionOption = features( - maven().groupId("org.apache.unomi").artifactId("unomi-distribution").versionAsInProject().type("xml").classifier("features"), - "unomi-distribution-opensearch-graphql" - ); - } else { - throw new IllegalArgumentException("Unknown search engine: " + searchEngine); + LOGGER.info("Persistence provider: {}", searchEngine); + System.out.println("Persistence provider: " + searchEngine); } + PersistenceITBackend backend = getPersistenceBackend(); + Option[] backendFeatures = backend.featureOptions(); + Option[] backendConfig = backend.configurationOptions(); + Option[] options = new Option[]{ - replaceConfigurationFile("etc/org.apache.unomi.router.cfg", new File("src/test/resources/org.apache.unomi.router.cfg")), - replaceConfigurationFile("data/tmp/1-basic-test.csv", new File("src/test/resources/1-basic-test.csv")), - replaceConfigurationFile("data/tmp/recurrent_import/2-surfers-test.csv", new File("src/test/resources/2-surfers-test.csv")), - replaceConfigurationFile("data/tmp/recurrent_import/3-surfers-overwrite-test.csv", new File("src/test/resources/3-surfers-overwrite-test.csv")), - replaceConfigurationFile("data/tmp/recurrent_import/4-surfers-delete-test.csv", new File("src/test/resources/4-surfers-delete-test.csv")), - replaceConfigurationFile("data/tmp/recurrent_import/5-ranking-test.csv", new File("src/test/resources/5-ranking-test.csv")), - replaceConfigurationFile("data/tmp/recurrent_import/6-actors-test.csv", new File("src/test/resources/6-actors-test.csv")), - replaceConfigurationFile("data/tmp/testLogin.json", new File("src/test/resources/testLogin.json")), - replaceConfigurationFile("data/tmp/testCopyProperties.json", new File("src/test/resources/testCopyProperties.json")), - replaceConfigurationFile("data/tmp/testCopyPropertiesWithoutSystemTags.json", new File("src/test/resources/testCopyPropertiesWithoutSystemTags.json")), - replaceConfigurationFile("data/tmp/testLoginEventCondition.json", new File("src/test/resources/testLoginEventCondition.json")), - replaceConfigurationFile("data/tmp/testClickEventCondition.json", new File("src/test/resources/testClickEventCondition.json")), - replaceConfigurationFile("data/tmp/testRuleGroovyAction.json", new File("src/test/resources/testRuleGroovyAction.json")), - replaceConfigurationFile("data/tmp/conditions/testIdsConditionLegacy.json", new File("src/test/resources/conditions/testIdsConditionLegacy.json")), - replaceConfigurationFile("data/tmp/conditions/testIdsConditionNew.json", new File("src/test/resources/conditions/testIdsConditionNew.json")), - replaceConfigurationFile("data/tmp/conditions/testBooleanConditionLegacy.json", new File("src/test/resources/conditions/testBooleanConditionLegacy.json")), - replaceConfigurationFile("data/tmp/conditions/testPropertyConditionLegacy.json", new File("src/test/resources/conditions/testPropertyConditionLegacy.json")), - replaceConfigurationFile("data/tmp/groovy/UpdateAddressAction.groovy", new File("src/test/resources/groovy/UpdateAddressAction.groovy")), + replaceConfigurationFile("etc/org.apache.unomi.router.cfg", resolveTestResource("src/test/resources/org.apache.unomi.router.cfg")), + replaceConfigurationFile("data/tmp/1-basic-test.csv", resolveTestResource("src/test/resources/1-basic-test.csv")), + replaceConfigurationFile("data/tmp/recurrent_import/2-surfers-test.csv", resolveTestResource("src/test/resources/2-surfers-test.csv")), + replaceConfigurationFile("data/tmp/recurrent_import/3-surfers-overwrite-test.csv", resolveTestResource("src/test/resources/3-surfers-overwrite-test.csv")), + replaceConfigurationFile("data/tmp/recurrent_import/4-surfers-delete-test.csv", resolveTestResource("src/test/resources/4-surfers-delete-test.csv")), + replaceConfigurationFile("data/tmp/recurrent_import/5-ranking-test.csv", resolveTestResource("src/test/resources/5-ranking-test.csv")), + replaceConfigurationFile("data/tmp/recurrent_import/6-actors-test.csv", resolveTestResource("src/test/resources/6-actors-test.csv")), + replaceConfigurationFile("data/tmp/testLogin.json", resolveTestResource("src/test/resources/testLogin.json")), + replaceConfigurationFile("data/tmp/testCopyProperties.json", resolveTestResource("src/test/resources/testCopyProperties.json")), + replaceConfigurationFile("data/tmp/testCopyPropertiesWithoutSystemTags.json", resolveTestResource("src/test/resources/testCopyPropertiesWithoutSystemTags.json")), + replaceConfigurationFile("data/tmp/testLoginEventCondition.json", resolveTestResource("src/test/resources/testLoginEventCondition.json")), + replaceConfigurationFile("data/tmp/testClickEventCondition.json", resolveTestResource("src/test/resources/testClickEventCondition.json")), + replaceConfigurationFile("data/tmp/testRuleGroovyAction.json", resolveTestResource("src/test/resources/testRuleGroovyAction.json")), + replaceConfigurationFile("data/tmp/conditions/testIdsConditionLegacy.json", resolveTestResource("src/test/resources/conditions/testIdsConditionLegacy.json")), + replaceConfigurationFile("data/tmp/conditions/testIdsConditionNew.json", resolveTestResource("src/test/resources/conditions/testIdsConditionNew.json")), + replaceConfigurationFile("data/tmp/conditions/testBooleanConditionLegacy.json", resolveTestResource("src/test/resources/conditions/testBooleanConditionLegacy.json")), + replaceConfigurationFile("data/tmp/conditions/testPropertyConditionLegacy.json", resolveTestResource("src/test/resources/conditions/testPropertyConditionLegacy.json")), + replaceConfigurationFile("data/tmp/groovy/UpdateAddressAction.groovy", resolveTestResource("src/test/resources/groovy/UpdateAddressAction.groovy")), editConfigurationFilePut("etc/org.ops4j.pax.logging.cfg", "log4j2.rootLogger.level", "INFO"), editConfigurationFilePut("etc/org.apache.karaf.features.cfg", "serviceRequirements", "disable"), editConfigurationFilePut("etc/system.properties", "my.system.property", System.getProperty("my.system.property")), - editConfigurationFilePut("etc/system.properties", SEARCH_ENGINE_PROPERTY, System.getProperty(SEARCH_ENGINE_PROPERTY, SEARCH_ENGINE_ELASTICSEARCH)), - editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.elasticsearch.cluster.name", "contextElasticSearchITests"), - editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.elasticsearch.addresses", "localhost:" + getSearchPort()), - editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.elasticsearch.taskWaitingPollingInterval", "50"), - editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.elasticsearch.rollover.maxDocs", "300"), - editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.elasticsearch.minimalClusterState", "YELLOW"), - editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.opensearch.cluster.name", "contextElasticSearchITests"), - editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.opensearch.addresses", "localhost:" + getSearchPort()), - editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.opensearch.username", "admin"), - editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.opensearch.password", "Unomi.1ntegrat10n.Tests"), - editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.opensearch.sslEnable", "false"), - editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.opensearch.sslTrustAllCertificates", "true"), - editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.opensearch.rollover.maxDocs", "300"), - editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.opensearch.minimalClusterState", "YELLOW"), + editConfigurationFilePut("etc/system.properties", SEARCH_ENGINE_PROPERTY, searchEngine), + editConfigurationFilePut("etc/system.properties", PERSISTENCE_PROVIDER_PROPERTY, searchEngine), editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.migration.tenant.id", TEST_TENANT_ID), // Default scheduler.thread.poolSize (5) is sized for the near-instant in-memory unit-test // double, not a real ES/OS backend. Under real refresh/write latency, the checker, task @@ -680,9 +682,6 @@ public abstract class BaseIT extends KarafTestSupport { systemProperty("org.ops4j.pax.exam.rbc.rmi.port").value("1199"), systemProperty("org.apache.unomi.healthcheck.enabled").value("true"), - featuresOption, // Add the features option - distributionOption, // Add the distribution option - configureConsole().startRemoteShell(), logLevel(LogLevel.INFO), keepRuntimeFolder(), @@ -691,6 +690,8 @@ public abstract class BaseIT extends KarafTestSupport { }; List<Option> karafOptions = new ArrayList<>(); karafOptions.addAll(Arrays.asList(options)); + karafOptions.addAll(Arrays.asList(backendConfig)); + karafOptions.addAll(Arrays.asList(backendFeatures)); String karafDebug = System.getProperty("it.karaf.debug"); if (karafDebug != null) { @@ -1435,6 +1436,10 @@ public abstract class BaseIT extends KarafTestSupport { if (searchEngineConfiguredForTesting) { return; } + if (!persistenceCapabilities().httpAdminApi()) { + searchEngineConfiguredForTesting = true; + return; + } try (CloseableHttpClient client = createSearchEngineHttpClient()) { String baseUrl = getSearchEngineBaseUrl(); ensureZeroReplicaIndexTemplate(client, baseUrl); @@ -1464,6 +1469,9 @@ public abstract class BaseIT extends KarafTestSupport { } private void enforceZeroReplicasAndWaitForCluster(String context) { + if (!persistenceCapabilities().httpAdminApi()) { + return; + } try (CloseableHttpClient client = createSearchEngineHttpClient()) { String baseUrl = getSearchEngineBaseUrl(); ensureZeroReplicaIndexTemplate(client, baseUrl); @@ -1517,6 +1525,14 @@ public abstract class BaseIT extends KarafTestSupport { * Stage 3 (post-Unomi start): assert cluster health for single-node IT (ES and OpenSearch). */ protected void assertClusterHealthy(String context) { + if (!persistenceCapabilities().httpAdminApi()) { + try { + getPersistenceBackend().assertHealthyAfterUnomiStart(); + } catch (Exception e) { + throw new IllegalStateException("Backend health assertion failed: " + context, e); + } + return; + } try (CloseableHttpClient client = createSearchEngineHttpClient()) { String baseUrl = getSearchEngineBaseUrl(); ensureZeroReplicaIndexTemplate(client, baseUrl); @@ -1550,36 +1566,31 @@ public abstract class BaseIT extends KarafTestSupport { } protected static String getSearchEngineBaseUrl() { - if (SEARCH_ENGINE_OPENSEARCH.equals(searchEngine)) { - return "http://localhost:"; + getSearchPort(); - } - return "http://localhost:"; + getSearchPort(); + requireHttpAdminApi("getSearchEngineBaseUrl"); + return getPersistenceBackend().searchBaseUrl(); } protected CloseableHttpClient createSearchEngineHttpClient() throws IOException { - if (SEARCH_ENGINE_OPENSEARCH.equals(searchEngine)) { - BasicCredentialsProvider credentialsProvider = new BasicCredentialsProvider(); - credentialsProvider.setCredentials(AuthScope.ANY, - new UsernamePasswordCredentials("admin", "Unomi.1ntegrat10n.Tests")); - return HttpUtils.initHttpClient(true, credentialsProvider); - } - return HttpUtils.initHttpClient(true, null); + requireHttpAdminApi("createSearchEngineHttpClient"); + return getPersistenceBackend().createSearchHttpClient(); } /** - * Gets the appropriate search engine port based on the configured search engine. + * Gets the appropriate search engine port based on the configured persistence provider. * * @return The port number as a string */ protected static String getSearchPort() { - String searchEngine = System.getProperty(SEARCH_ENGINE_PROPERTY, SEARCH_ENGINE_ELASTICSEARCH); - if (SEARCH_ENGINE_OPENSEARCH.equals(searchEngine)) { - // For OpenSearch, get the port from the system property set by maven-failsafe-plugin - return System.getProperty("org.apache.unomi.opensearch.addresses", "localhost:9401") - .split(":")[1]; // Extract port number from "localhost:9401" - } else { - // For Elasticsearch, use the default port or system property if set - return System.getProperty("elasticsearch.port", "9400"); + requireHttpAdminApi("getSearchPort"); + return getPersistenceBackend().searchPort(); + } + + private static void requireHttpAdminApi(String caller) { + PersistenceITBackend backend = getPersistenceBackend(); + if (!backend.capabilities().httpAdminApi()) { + throw new UnsupportedOperationException( + caller + " requires httpAdminApi, but provider '" + backend.providerId() + + "' does not advertise it (skip via Assume / SearchBackendIT / CorePersistenceITs)"); } } diff --git a/itests/src/test/java/org/apache/unomi/itests/AllITs.java b/itests/src/test/java/org/apache/unomi/itests/CorePersistenceITs.java similarity index 85% copy from itests/src/test/java/org/apache/unomi/itests/AllITs.java copy to itests/src/test/java/org/apache/unomi/itests/CorePersistenceITs.java index 8617b1dcb..6cc692a0d 100644 --- a/itests/src/test/java/org/apache/unomi/itests/AllITs.java +++ b/itests/src/test/java/org/apache/unomi/itests/CorePersistenceITs.java @@ -24,9 +24,12 @@ import org.junit.runner.RunWith; import org.junit.runners.Suite.SuiteClasses; /** - * Defines suite of test classes to run. - * - * @author Sergiy Shyrkov + * Full behavioural IT suite for any {@code PersistenceService} provider. + * <p> + * Membership matches {@link AllITs}. Tests that need HTTP admin / snapshot / rollover + * APIs use {@link org.junit.Assume} on {@link org.apache.unomi.itests.persistence.PersistenceITCapabilities} + * so unsupported backends <em>skip</em> (not fail, not false-pass). Prefer this suite for + * PostgreSQL, in-memory, JDBC, etc.; Elasticsearch / OpenSearch CI may keep using {@link AllITs}. */ @RunWith(ProgressSuite.class) @SuiteClasses({ @@ -83,5 +86,5 @@ import org.junit.runners.Suite.SuiteClasses; RolloverIT.class, HealthCheckIT.class }) -public class AllITs { +public class CorePersistenceITs { } diff --git a/itests/src/test/java/org/apache/unomi/itests/HealthCheckIT.java b/itests/src/test/java/org/apache/unomi/itests/HealthCheckIT.java index 9fe11829d..305c2f38e 100644 --- a/itests/src/test/java/org/apache/unomi/itests/HealthCheckIT.java +++ b/itests/src/test/java/org/apache/unomi/itests/HealthCheckIT.java @@ -41,7 +41,12 @@ import java.util.concurrent.*; import static org.junit.Assert.fail; /** - * Health Check Integration Tests + * Health Check Integration Tests. + * <p> + * Always asserts {@code karaf}, {@code unomi}, and {@code persistence} LIVE. + * Optional probes ({@code providerNamedHealthProbe}, {@code clusterHealthProbe}) are + * asserted only when the active {@link org.apache.unomi.itests.persistence.PersistenceITCapabilities} + * advertise them — so the same class covers Elasticsearch, OpenSearch, PostgreSQL, etc. */ @RunWith(PaxExam.class) @ExamReactorStrategy(PerSuite.class) @@ -57,15 +62,10 @@ public class HealthCheckIT extends BaseIT { public void testHealthCheck() { try { List<HealthCheckResponse> response = get(HEALTHCHECK_ENDPOINT, new TypeReference<>() {}); - LOGGER.info("configured search engine: {}", searchEngine); + LOGGER.info("configured persistence provider: {}", getPersistenceBackend().providerId()); LOGGER.info("health check response: {}", response); Assert.assertNotNull(response); - Assert.assertEquals(5, response.size()); - Assert.assertTrue(response.stream().anyMatch(r -> r.getName().equals("karaf") && r.getStatus() == HealthCheckResponse.Status.LIVE)); - Assert.assertTrue(response.stream().anyMatch(r -> r.getName().equals(searchEngine) && r.getStatus() == HealthCheckResponse.Status.LIVE)); - Assert.assertTrue(response.stream().anyMatch(r -> r.getName().equals("unomi") && r.getStatus() == HealthCheckResponse.Status.LIVE)); - Assert.assertTrue(response.stream().anyMatch(r -> r.getName().equals("persistence") && r.getStatus() == HealthCheckResponse.Status.LIVE)); - Assert.assertTrue(response.stream().anyMatch(r -> r.getName().equals("cluster") && r.getStatus() == HealthCheckResponse.Status.LIVE)); + assertHealthCheckLive(response); } catch (Exception e) { LOGGER.error("Error while executing health check", e); fail("Error while executing health check" + e.getMessage()); @@ -88,12 +88,7 @@ public class HealthCheckIT extends BaseIT { } for (Future<List<HealthCheckResponse>> future : futures) { List<HealthCheckResponse> health = future.get(10, TimeUnit.SECONDS); - Assert.assertEquals(5, health.size()); - Assert.assertTrue(health.stream().anyMatch(r -> r.getName().equals("karaf") && r.getStatus() == HealthCheckResponse.Status.LIVE)); - Assert.assertTrue(health.stream().anyMatch(r -> r.getName().equals(searchEngine) && r.getStatus() == HealthCheckResponse.Status.LIVE)); - Assert.assertTrue(health.stream().anyMatch(r -> r.getName().equals("unomi") && r.getStatus() == HealthCheckResponse.Status.LIVE)); - Assert.assertTrue(health.stream().anyMatch(r -> r.getName().equals("persistence") && r.getStatus() == HealthCheckResponse.Status.LIVE)); - Assert.assertTrue(health.stream().anyMatch(r -> r.getName().equals("cluster") && r.getStatus() == HealthCheckResponse.Status.LIVE)); + assertHealthCheckLive(health); } Thread.sleep(10); } @@ -109,6 +104,34 @@ public class HealthCheckIT extends BaseIT { } } + private void assertHealthCheckLive(List<HealthCheckResponse> response) { + Assert.assertNotNull(response); + Assert.assertTrue(response.stream().anyMatch(r -> r.getName().equals("karaf") && r.getStatus() == HealthCheckResponse.Status.LIVE)); + Assert.assertTrue(response.stream().anyMatch(r -> r.getName().equals("unomi") && r.getStatus() == HealthCheckResponse.Status.LIVE)); + Assert.assertTrue(response.stream().anyMatch(r -> r.getName().equals("persistence") && r.getStatus() == HealthCheckResponse.Status.LIVE)); + int expectedMinProbes = 3; // karaf, unomi, persistence + boolean named = persistenceCapabilities().providerNamedHealthProbe(); + boolean cluster = persistenceCapabilities().clusterHealthProbe(); + if (named) { + expectedMinProbes++; + String namedProbe = getPersistenceBackend().providerId(); + Assert.assertTrue(response.stream().anyMatch(r -> r.getName().equals(namedProbe) && r.getStatus() == HealthCheckResponse.Status.LIVE)); + } + if (cluster) { + expectedMinProbes++; + Assert.assertTrue(response.stream().anyMatch(r -> r.getName().equals("cluster") && r.getStatus() == HealthCheckResponse.Status.LIVE)); + } + // ES/OS advertise both optional probes → keep historical exact size (5). + // Other backends may expose extra probes; require at least the expected set. + if (named && cluster) { + Assert.assertEquals("Unexpected health probe count: " + response, expectedMinProbes, response.size()); + } else { + Assert.assertTrue( + "Expected at least " + expectedMinProbes + " health probes, got " + response.size() + ": " + response, + response.size() >= expectedMinProbes); + } + } + protected <T> T get(final String url, TypeReference<T> typeReference) { CloseableHttpResponse response = null; try { diff --git a/itests/src/test/java/org/apache/unomi/itests/JSONSchemaIT.java b/itests/src/test/java/org/apache/unomi/itests/JSONSchemaIT.java index e37c23cc8..e9fcd7c37 100644 --- a/itests/src/test/java/org/apache/unomi/itests/JSONSchemaIT.java +++ b/itests/src/test/java/org/apache/unomi/itests/JSONSchemaIT.java @@ -25,6 +25,7 @@ import org.apache.http.util.EntityUtils; import org.apache.unomi.api.Event; import org.apache.unomi.api.Scope; import org.apache.unomi.api.conditions.Condition; +import org.apache.unomi.itests.persistence.PersistenceITCapabilities; import org.apache.unomi.itests.tools.LogChecker; import org.apache.unomi.itests.tools.httpclient.HttpClientThatWaitsForUnomi; import org.apache.unomi.schema.api.JsonSchemaWrapper; @@ -377,16 +378,14 @@ public class JSONSchemaIT extends BaseIT { // Refresh to ensure event is queryable refreshPersistence(Event.class); final Condition finalCondition = condition; - // For Elasticsearch, range queries on flattened properties should return null or empty list - // For OpenSearch, they may return results + // Providers differ on range queries against flattened properties (hits vs empty). // We just need to wait for the query to execute (not throw an exception) refreshPersistence(Event.class); org.apache.unomi.api.PartialList<Event> queryResult = persistenceService.query(finalCondition, null, Event.class, 0, -1); - if ("opensearch".equals(searchEngine)) { - assertNotNull("OpenSearch should return results for flattened properties", queryResult); + if (persistenceCapabilities().flattenedRangeQueryResult() == PersistenceITCapabilities.FlattenedRangeQueryResult.HITS) { + assertNotNull("provider that returns flattened range hits should return results", queryResult); } else { - // Elasticsearch should return null or empty list for range queries on flattened properties - assertTrue("Elasticsearch should return null or empty list for flattened properties range query", + assertTrue("provider that returns empty for flattened range should return null or empty list", queryResult == null || queryResult.getList() == null || queryResult.getList().isEmpty()); } diff --git a/itests/src/test/java/org/apache/unomi/itests/ProfileServiceIT.java b/itests/src/test/java/org/apache/unomi/itests/ProfileServiceIT.java index dff72a1dc..e256c7004 100644 --- a/itests/src/test/java/org/apache/unomi/itests/ProfileServiceIT.java +++ b/itests/src/test/java/org/apache/unomi/itests/ProfileServiceIT.java @@ -153,7 +153,7 @@ public class ProfileServiceIT extends BaseIT { @Test public void testGetProfileWithWrongScrollerIdThrowException() throws InterruptedException, IOException { boolean throwExceptionCurrent = false; - Configuration searchEngineConfiguration = configurationAdmin.getConfiguration("org.apache.unomi.persistence." + searchEngine); + Configuration searchEngineConfiguration = configurationAdmin.getConfiguration(persistenceConfigPid()); if (searchEngineConfiguration != null && searchEngineConfiguration.getProperties().get("throwExceptions") != null) { try { if (searchEngineConfiguration.getProperties().get("throwExceptions") instanceof String) { @@ -167,7 +167,7 @@ public class ProfileServiceIT extends BaseIT { } } - updateConfiguration(null, "org.apache.unomi.persistence." + searchEngine, "throwExceptions", true); + updateConfiguration(null, persistenceConfigPid(), "throwExceptions", true); Query query = new Query(); query.setLimit(2); @@ -180,7 +180,7 @@ public class ProfileServiceIT extends BaseIT { } catch (RuntimeException ex) { // Should get here since this scenario should throw exception } finally { - updateConfiguration(null, "org.apache.unomi.persistence." + searchEngine, "throwExceptions", + updateConfiguration(null, persistenceConfigPid(), "throwExceptions", throwExceptionCurrent); } } diff --git a/itests/src/test/java/org/apache/unomi/itests/ProfileServiceWithoutOverwriteIT.java b/itests/src/test/java/org/apache/unomi/itests/ProfileServiceWithoutOverwriteIT.java index 2f94dd198..922b08bf9 100644 --- a/itests/src/test/java/org/apache/unomi/itests/ProfileServiceWithoutOverwriteIT.java +++ b/itests/src/test/java/org/apache/unomi/itests/ProfileServiceWithoutOverwriteIT.java @@ -44,8 +44,7 @@ public class ProfileServiceWithoutOverwriteIT extends BaseIT { @Configuration public Option[] config() { - - searchEngine = System.getProperty(SEARCH_ENGINE_PROPERTY, SEARCH_ENGINE_ELASTICSEARCH); + resolvePersistenceBackend(); System.out.println("Search Engine: " + searchEngine); List<Option> options = new ArrayList<>(); diff --git a/itests/src/test/java/org/apache/unomi/itests/RolloverIT.java b/itests/src/test/java/org/apache/unomi/itests/RolloverIT.java index 449bfa278..667070dde 100644 --- a/itests/src/test/java/org/apache/unomi/itests/RolloverIT.java +++ b/itests/src/test/java/org/apache/unomi/itests/RolloverIT.java @@ -18,8 +18,12 @@ package org.apache.unomi.itests; import com.fasterxml.jackson.databind.JsonNode; import org.apache.http.impl.client.CloseableHttpClient; +import org.apache.unomi.itests.persistence.PersistenceITCapabilities; +import org.apache.unomi.itests.persistence.SearchBackendIT; import org.apache.unomi.shell.migration.utils.HttpUtils; +import org.junit.Assume; import org.junit.Test; +import org.junit.experimental.categories.Category; import org.junit.runner.RunWith; import org.ops4j.pax.exam.junit.PaxExam; import org.ops4j.pax.exam.spi.reactors.ExamReactorStrategy; @@ -31,6 +35,7 @@ import java.util.Map; import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; /** * Verifies that Unomi correctly wires up the event index's rollover lifecycle on both Elasticsearch (ILM) @@ -52,9 +57,14 @@ import static org.junit.Assert.assertTrue; * that happens ISM has nothing left to transition to (the rollover policy's single state has no further * transitions) and marks that now-rolled-over index's management as completed/disabled, which would make a * hardcoded context-event-000001 check fail even though the rollover machinery worked correctly. + * <p> + * Non-search providers (e.g. PostgreSQL) leave {@link PersistenceITCapabilities.IndexRolloverApi#NONE} + * so this test is {@link Assume Assumed} skipped (not failed). Kept in {@link CorePersistenceITs} + * / {@link AllITs} for maximum suite coverage across backends. */ @RunWith(PaxExam.class) @ExamReactorStrategy(PerSuite.class) +@Category(SearchBackendIT.class) public class RolloverIT extends BaseIT { private static final String EVENT_ALIAS = "context-event"; @@ -63,16 +73,36 @@ public class RolloverIT extends BaseIT { @Test public void testEventIndexRolloverIsProperlyConfigured() throws Exception { + PersistenceITCapabilities caps = persistenceCapabilities(); + Assume.assumeTrue( + "Index rollover requires an HTTP rollover API (provider=" + + getPersistenceBackend().providerId() + ")", + caps.indexRolloverApi().isPresent()); + Assume.assumeTrue( + "Index rollover assertions require httpAdminApi (provider=" + + getPersistenceBackend().providerId() + ")", + caps.httpAdminApi()); + try (CloseableHttpClient client = createSearchEngineHttpClient()) { String writeIndex = resolveCurrentWriteIndex(client); JsonNode indexRoot = getJson(client, "/" + writeIndex + "/_settings?flat_settings=true").get(writeIndex); assertTrue("Expected the event write index " + writeIndex + " to already exist", indexRoot != null); JsonNode settings = indexRoot.get("settings"); - if (SEARCH_ENGINE_OPENSEARCH.equals(searchEngine)) { - assertOpenSearchRolloverConfigured(client, settings, writeIndex); - } else { - assertElasticsearchRolloverConfigured(client, settings, writeIndex); + switch (caps.indexRolloverApi()) { + case STATE_MANAGEMENT: + assertStateManagementRolloverConfigured(client, settings, writeIndex); + break; + case LIFECYCLE: + assertLifecycleRolloverConfigured(client, settings, writeIndex); + break; + case NONE: + fail("unreachable: indexRolloverApi was assumed present"); + break; + default: { + PersistenceITCapabilities.IndexRolloverApi unexpected = caps.indexRolloverApi(); + throw new IllegalStateException("Unhandled IndexRolloverApi: " + unexpected); + } } } } @@ -94,7 +124,7 @@ public class RolloverIT extends BaseIT { throw new AssertionError("Could not find a write index for alias " + EVENT_ALIAS); } - private void assertOpenSearchRolloverConfigured(CloseableHttpClient client, JsonNode settings, String writeIndex) throws IOException { + private void assertStateManagementRolloverConfigured(CloseableHttpClient client, JsonNode settings, String writeIndex) throws IOException { assertEquals("event index should reference the Unomi rollover policy", POLICY_ID, text(settings, "index.plugins.index_state_management.policy_id")); assertEquals("event index rollover_alias should be the event write alias", @@ -116,7 +146,7 @@ public class RolloverIT extends BaseIT { assertEquals(EXPECTED_MAX_DOCS, rolloverAction.get("min_doc_count").asLong()); } - private void assertElasticsearchRolloverConfigured(CloseableHttpClient client, JsonNode settings, String writeIndex) throws IOException { + private void assertLifecycleRolloverConfigured(CloseableHttpClient client, JsonNode settings, String writeIndex) throws IOException { assertEquals("event index should reference the Unomi rollover policy", POLICY_ID, text(settings, "index.lifecycle.name")); assertEquals("event index rollover_alias should be the event write alias", diff --git a/itests/src/test/java/org/apache/unomi/itests/migration/Migrate16xToCurrentVersionIT.java b/itests/src/test/java/org/apache/unomi/itests/migration/Migrate16xToCurrentVersionIT.java index 214f71fa8..bd03f8436 100644 --- a/itests/src/test/java/org/apache/unomi/itests/migration/Migrate16xToCurrentVersionIT.java +++ b/itests/src/test/java/org/apache/unomi/itests/migration/Migrate16xToCurrentVersionIT.java @@ -26,19 +26,23 @@ import org.apache.unomi.api.conditions.ConditionType; import org.apache.unomi.api.tenants.Tenant; import org.apache.unomi.geonames.services.GeonameEntry; import org.apache.unomi.itests.BaseIT; +import org.apache.unomi.itests.persistence.SearchBackendIT; import org.apache.unomi.persistence.spi.aggregate.TermsAggregate; import org.apache.unomi.shell.migration.utils.HttpUtils; import org.apache.unomi.shell.migration.utils.MigrationUtils; import org.junit.After; import org.junit.Assert; +import org.junit.Assume; import org.junit.Before; import org.junit.Test; +import org.junit.experimental.categories.Category; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import java.io.IOException; import java.util.*; +@Category(SearchBackendIT.class) public class Migrate16xToCurrentVersionIT extends BaseIT { private final static Logger LOGGER = LoggerFactory.getLogger(Migrate16xToCurrentVersionIT.class); @@ -54,7 +58,7 @@ public class Migrate16xToCurrentVersionIT extends BaseIT { // Elasticsearch connection constants private static String getEsBaseUrl() { - return "http://localhost:"; + getSearchPort(); + return getSearchEngineBaseUrl(); } private static String getEsSnapshotRepo() { return getEsBaseUrl() + "/_snapshot/snapshots_repository/"; @@ -109,8 +113,9 @@ public class Migrate16xToCurrentVersionIT extends BaseIT { // This is called from BaseIT and will run before any migration setup checkSearchEngine(); - if (SEARCH_ENGINE_OPENSEARCH.equals(searchEngine)) { - System.out.println("Migration from 1.x to 2.x not supported for OpenSearch, skipping snapshot restore"); + if (!persistenceCapabilities().snapshotRestoreMigration()) { + System.out.println("snapshotRestoreMigration not supported for provider " + + getPersistenceBackend().providerId() + " — starting Unomi without legacy snapshot restore"); super.waitForStartup(); return; } @@ -119,7 +124,7 @@ public class Migrate16xToCurrentVersionIT extends BaseIT { LOGGER.info("Restoring snapshot into search engine..."); // Restore snapshot from 1.6.x - try (CloseableHttpClient httpClient = HttpUtils.initHttpClient(true, null)) { + try (CloseableHttpClient httpClient = createSearchEngineHttpClient()) { // Create snapshot repo HttpUtils.executePutRequest(httpClient, getEsSnapshotRepo(), resourceAsString(RESOURCE_CREATE_SNAPSHOTS_REPO), null); // Get snapshot, insure it exists @@ -199,10 +204,10 @@ public class Migrate16xToCurrentVersionIT extends BaseIT { */ @Test public void checkMigratedData() throws Exception { - if (SEARCH_ENGINE_OPENSEARCH.equals(searchEngine)) { - System.out.println("Migration from 1.x to 2.x not supported for OpenSearch, skipping checks"); - return; - } + Assume.assumeTrue( + "snapshotRestoreMigration not supported for provider " + + getPersistenceBackend().providerId(), + persistenceCapabilities().snapshotRestoreMigration()); checkMergedProfilesAliases(); checkProfileInterests(); checkProfileTotalNbOfVisits(); @@ -654,10 +659,10 @@ public class Migrate16xToCurrentVersionIT extends BaseIT { * Test that the default tenant was created during migration (migrate-3.1.0-10-tenantInitialization) */ private void checkDefaultTenantCreated() throws Exception { - if (SEARCH_ENGINE_OPENSEARCH.equals(searchEngine)) { - System.out.println("Migration from 1.x to 2.x not supported for OpenSearch, skipping checks"); - return; - } + Assume.assumeTrue( + "snapshotRestoreMigration not supported for provider " + + getPersistenceBackend().providerId(), + persistenceCapabilities().snapshotRestoreMigration()); // Check that the default tenant index exists Assert.assertTrue("Default tenant index should exist", MigrationUtils.indexExists(httpClient, getEsBaseUrl(), INDEX_PREFIX_CONTEXT + "tenant")); @@ -692,10 +697,10 @@ public class Migrate16xToCurrentVersionIT extends BaseIT { * have proper tenant information, audit metadata, and are accessible via definitionsService. */ private void checkDefinitionsServiceObjectsAccessible() throws Exception { - if (SEARCH_ENGINE_OPENSEARCH.equals(searchEngine)) { - System.out.println("Migration from 1.x to 2.x not supported for OpenSearch, skipping checks"); - return; - } + Assume.assumeTrue( + "snapshotRestoreMigration not supported for provider " + + getPersistenceBackend().providerId(), + persistenceCapabilities().snapshotRestoreMigration()); // Refresh the definitions service cache to ensure migrated items are loaded // This is necessary because items might be in persistence but not yet in cache @@ -811,10 +816,10 @@ public class Migrate16xToCurrentVersionIT extends BaseIT { * all condition types that use legacy *ESQueryBuilder syntax to use the new generic QueryBuilder syntax. */ private void checkLegacyQueryBuilderMigration() throws Exception { - if (SEARCH_ENGINE_OPENSEARCH.equals(searchEngine)) { - System.out.println("Migration from 1.x to 2.x not supported for OpenSearch, skipping checks"); - return; - } + Assume.assumeTrue( + "snapshotRestoreMigration not supported for provider " + + getPersistenceBackend().providerId(), + persistenceCapabilities().snapshotRestoreMigration()); // Refresh the definitions service cache to ensure migrated items are loaded definitionsService.refresh(); diff --git a/itests/src/test/java/org/apache/unomi/itests/persistence/ElasticsearchITBackend.java b/itests/src/test/java/org/apache/unomi/itests/persistence/ElasticsearchITBackend.java new file mode 100644 index 000000000..467c9d0fd --- /dev/null +++ b/itests/src/test/java/org/apache/unomi/itests/persistence/ElasticsearchITBackend.java @@ -0,0 +1,117 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License + */ +package org.apache.unomi.itests.persistence; + +import org.apache.http.impl.client.CloseableHttpClient; +import org.apache.unomi.shell.migration.utils.HttpUtils; +import org.ops4j.pax.exam.Option; + +import java.io.IOException; + +import static org.ops4j.pax.exam.CoreOptions.maven; +import static org.ops4j.pax.exam.karaf.options.KarafDistributionOption.editConfigurationFilePut; +import static org.ops4j.pax.exam.karaf.options.KarafDistributionOption.features; + +/** + * Built-in Elasticsearch backend for Unomi integration tests. + */ +public class ElasticsearchITBackend implements PersistenceITBackend { + + @Override + public String providerId() { + return PersistenceITBackendResolver.PROVIDER_ELASTICSEARCH; + } + + @Override + public Option[] featureOptions() { + return new Option[]{ + features( + maven().groupId("org.apache.unomi").artifactId("unomi-kar").versionAsInProject().type("xml").classifier("features"), + "unomi-base", + "unomi-startup", + "unomi-elasticsearch-core", + "unomi-persistence-core", + "unomi-services", + "unomi-cxs-privacy-extension-services", + "unomi-plugins-base", + "unomi-plugins-request", + "unomi-plugins-mail", + "unomi-plugins-optimization-test", + "unomi-rest-api", + "unomi-cxs-privacy-extension", + "unomi-elasticsearch-conditions", + "unomi-cxs-lists-extension", + "unomi-cxs-geonames-extension", + "unomi-shell-dev-commands", + "unomi-wab", + "unomi-web-tracker", + "unomi-healthcheck-elasticsearch", + "unomi-router-karaf-feature", + "unomi-groovy-actions", + "unomi-rest-ui", + "cdp-graphql-feature", + "unomi-startup-complete" + ), + features( + maven().groupId("org.apache.unomi").artifactId("unomi-distribution").versionAsInProject().type("xml").classifier("features"), + "unomi-distribution-elasticsearch-graphql" + ) + }; + } + + @Override + public Option[] configurationOptions() { + String port = searchPort(); + return new Option[]{ + editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.elasticsearch.cluster.name", "contextElasticSearchITests"), + editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.elasticsearch.addresses", "localhost:" + port), + editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.elasticsearch.taskWaitingPollingInterval", "50"), + editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.elasticsearch.rollover.maxDocs", "300"), + editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.elasticsearch.minimalClusterState", "YELLOW"), + }; + } + + @Override + public String distributionFeature() { + return "unomi-distribution-elasticsearch"; + } + + @Override + public String persistenceConfigPid() { + return "org.apache.unomi.persistence.elasticsearch"; + } + + @Override + public PersistenceITCapabilities capabilities() { + return PersistenceITCapabilities.elasticsearch(); + } + + @Override + public String searchBaseUrl() { + return "http://localhost:"; + searchPort(); + } + + @Override + public String searchPort() { + return System.getProperty("elasticsearch.port", "9400"); + } + + @Override + public CloseableHttpClient createSearchHttpClient() throws IOException { + return HttpUtils.initHttpClient(true, null); + } +} diff --git a/itests/src/test/java/org/apache/unomi/itests/persistence/OpenSearchITBackend.java b/itests/src/test/java/org/apache/unomi/itests/persistence/OpenSearchITBackend.java new file mode 100644 index 000000000..04e182264 --- /dev/null +++ b/itests/src/test/java/org/apache/unomi/itests/persistence/OpenSearchITBackend.java @@ -0,0 +1,130 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License + */ +package org.apache.unomi.itests.persistence; + +import org.apache.http.auth.AuthScope; +import org.apache.http.auth.UsernamePasswordCredentials; +import org.apache.http.impl.client.BasicCredentialsProvider; +import org.apache.http.impl.client.CloseableHttpClient; +import org.apache.unomi.shell.migration.utils.HttpUtils; +import org.ops4j.pax.exam.Option; + +import java.io.IOException; + +import static org.ops4j.pax.exam.CoreOptions.maven; +import static org.ops4j.pax.exam.karaf.options.KarafDistributionOption.editConfigurationFilePut; +import static org.ops4j.pax.exam.karaf.options.KarafDistributionOption.features; + +/** + * Built-in OpenSearch backend for Unomi integration tests. + */ +public class OpenSearchITBackend implements PersistenceITBackend { + + private static final String OS_USER = "admin"; + private static final String OS_PASSWORD = "Unomi.1ntegrat10n.Tests"; + + @Override + public String providerId() { + return PersistenceITBackendResolver.PROVIDER_OPENSEARCH; + } + + @Override + public Option[] featureOptions() { + return new Option[]{ + features( + maven().groupId("org.apache.unomi").artifactId("unomi-kar").versionAsInProject().type("xml").classifier("features"), + "unomi-base", + "unomi-startup", + "unomi-opensearch-core", + "unomi-persistence-core", + "unomi-services", + "unomi-cxs-privacy-extension-services", + "unomi-plugins-base", + "unomi-plugins-request", + "unomi-plugins-mail", + "unomi-plugins-optimization-test", + "unomi-rest-api", + "unomi-cxs-privacy-extension", + "unomi-opensearch-conditions", + "unomi-cxs-lists-extension", + "unomi-cxs-geonames-extension", + "unomi-shell-dev-commands", + "unomi-wab", + "unomi-web-tracker", + "unomi-healthcheck-opensearch", + "unomi-router-karaf-feature", + "unomi-groovy-actions", + "unomi-rest-ui", + "cdp-graphql-feature", + "unomi-startup-complete" + ), + features( + maven().groupId("org.apache.unomi").artifactId("unomi-distribution").versionAsInProject().type("xml").classifier("features"), + "unomi-distribution-opensearch-graphql" + ) + }; + } + + @Override + public Option[] configurationOptions() { + String port = searchPort(); + return new Option[]{ + editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.opensearch.cluster.name", "contextElasticSearchITests"), + editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.opensearch.addresses", "localhost:" + port), + editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.opensearch.username", OS_USER), + editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.opensearch.password", OS_PASSWORD), + editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.opensearch.sslEnable", "false"), + editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.opensearch.sslTrustAllCertificates", "true"), + editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.opensearch.rollover.maxDocs", "300"), + editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.opensearch.minimalClusterState", "YELLOW"), + }; + } + + @Override + public String distributionFeature() { + return "unomi-distribution-opensearch"; + } + + @Override + public String persistenceConfigPid() { + return "org.apache.unomi.persistence.opensearch"; + } + + @Override + public PersistenceITCapabilities capabilities() { + return PersistenceITCapabilities.opensearch(); + } + + @Override + public String searchBaseUrl() { + return "http://localhost:"; + searchPort(); + } + + @Override + public String searchPort() { + return System.getProperty("org.apache.unomi.opensearch.addresses", "localhost:9401") + .split(":")[1]; + } + + @Override + public CloseableHttpClient createSearchHttpClient() throws IOException { + BasicCredentialsProvider credentialsProvider = new BasicCredentialsProvider(); + credentialsProvider.setCredentials(AuthScope.ANY, + new UsernamePasswordCredentials(OS_USER, OS_PASSWORD)); + return HttpUtils.initHttpClient(true, credentialsProvider); + } +} diff --git a/itests/src/test/java/org/apache/unomi/itests/persistence/PersistenceITBackend.java b/itests/src/test/java/org/apache/unomi/itests/persistence/PersistenceITBackend.java new file mode 100644 index 000000000..50beaa657 --- /dev/null +++ b/itests/src/test/java/org/apache/unomi/itests/persistence/PersistenceITBackend.java @@ -0,0 +1,92 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License + */ +package org.apache.unomi.itests.persistence; + +import org.apache.http.impl.client.CloseableHttpClient; +import org.osgi.framework.BundleContext; +import org.osgi.service.cm.ConfigurationAdmin; +import org.ops4j.pax.exam.Option; + +import java.io.IOException; + +/** + * Test-only contract for provisioning and awaiting a {@code PersistenceService} provider + * during Pax Exam integration tests. + * <p> + * Built-in implementations cover Elasticsearch and OpenSearch. Additional providers register + * via {@link java.util.ServiceLoader} or {@code -Dunomi.persistence.it.backend=<fqcn>}. + */ +public interface PersistenceITBackend { + + /** Provider id (e.g. {@code elasticsearch}, {@code opensearch}). */ + String providerId(); + + /** + * Karaf feature options for this provider (kar features + distribution feature). + */ + Option[] featureOptions(); + + /** + * Provider-specific {@code etc/custom.system.properties} (and related) Pax Exam options. + */ + Option[] configurationOptions(); + + /** Distribution feature name passed to {@code unomi:setup -d=…}. */ + String distributionFeature(); + + /** + * Wait until the backend is usable before Unomi starts (cluster health, JDBC ping, …). + * Default is a no-op; search backends typically poll HTTP health from {@code BaseIT}. + * <p> + * Called from the Pax Exam driver / early {@code @Before} path — do not assume OSGi + * services are available yet. Prefer {@link #prepareBeforeUnomiSetup} for ConfigAdmin work. + */ + default void awaitBackendReady() throws Exception { + } + + /** + * Called in the Karaf JVM after {@code UnomiManagementService} is available and before + * {@code unomi:setup}. Use for ConfigAdmin DataSource patching, bundle restarts, etc. + */ + default void prepareBeforeUnomiSetup(BundleContext bundleContext, ConfigurationAdmin configurationAdmin) + throws Exception { + } + + /** Optional verification after Unomi has started. */ + default void assertHealthyAfterUnomiStart() throws Exception { + } + + /** ConfigAdmin PID for {@code org.apache.unomi.persistence.*} knobs (e.g. {@code throwExceptions}). */ + String persistenceConfigPid(); + + PersistenceITCapabilities capabilities(); + + /** HTTP base URL when {@link PersistenceITCapabilities#httpAdminApi()} is true. */ + default String searchBaseUrl() { + throw new UnsupportedOperationException(providerId() + " does not expose an HTTP admin API"); + } + + /** Search HTTP listen port when {@link PersistenceITCapabilities#httpAdminApi()} is true. */ + default String searchPort() { + throw new UnsupportedOperationException(providerId() + " does not expose an HTTP admin API"); + } + + /** HTTP client for admin APIs (may include basic auth). */ + default CloseableHttpClient createSearchHttpClient() throws IOException { + throw new UnsupportedOperationException(providerId() + " does not expose an HTTP admin API"); + } +} diff --git a/itests/src/test/java/org/apache/unomi/itests/persistence/PersistenceITBackendResolver.java b/itests/src/test/java/org/apache/unomi/itests/persistence/PersistenceITBackendResolver.java new file mode 100644 index 000000000..5c8466237 --- /dev/null +++ b/itests/src/test/java/org/apache/unomi/itests/persistence/PersistenceITBackendResolver.java @@ -0,0 +1,110 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License + */ +package org.apache.unomi.itests.persistence; + +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.ServiceLoader; + +/** + * Resolves the active {@link PersistenceITBackend} for the IT harness. + * <p> + * Resolution order: + * <ol> + * <li>{@code -Dunomi.persistence.it.backend=<fully.qualified.ClassName>}</li> + * <li>{@link ServiceLoader} implementations matching the provider id</li> + * <li>Built-in Elasticsearch / OpenSearch backends</li> + * </ol> + * Provider id comes from {@code unomi.persistence.provider}, falling back to the + * deprecated {@code unomi.search.engine} alias (default {@code elasticsearch}). + */ +public final class PersistenceITBackendResolver { + + public static final String PROVIDER_PROPERTY = "unomi.persistence.provider"; + /** @deprecated use {@link #PROVIDER_PROPERTY}; kept for existing scripts and CI. */ + @Deprecated + public static final String SEARCH_ENGINE_PROPERTY = "unomi.search.engine"; + public static final String BACKEND_CLASS_PROPERTY = "unomi.persistence.it.backend"; + + public static final String PROVIDER_ELASTICSEARCH = "elasticsearch"; + public static final String PROVIDER_OPENSEARCH = "opensearch"; + + private PersistenceITBackendResolver() { + } + + public static String resolveProviderId() { + String provider = System.getProperty(PROVIDER_PROPERTY); + if (provider != null && !provider.isBlank()) { + return provider.trim(); + } + return System.getProperty(SEARCH_ENGINE_PROPERTY, PROVIDER_ELASTICSEARCH).trim(); + } + + public static PersistenceITBackend resolve() { + String explicitClass = System.getProperty(BACKEND_CLASS_PROPERTY); + if (explicitClass != null && !explicitClass.isBlank()) { + return instantiate(explicitClass.trim()); + } + + String providerId = resolveProviderId(); + Map<String, PersistenceITBackend> byId = loadBackendsById(); + PersistenceITBackend backend = byId.get(providerId); + if (backend != null) { + return backend; + } + + throw new IllegalArgumentException( + "No PersistenceITBackend registered for provider '" + providerId + "'. " + + "Set -D" + PROVIDER_PROPERTY + "=<id> (or deprecated -D" + SEARCH_ENGINE_PROPERTY + "), " + + "register a ServiceLoader implementation, or set -D" + BACKEND_CLASS_PROPERTY + "=<fqcn>. " + + "Known built-in / discovered ids: " + byId.keySet()); + } + + private static Map<String, PersistenceITBackend> loadBackendsById() { + Map<String, PersistenceITBackend> byId = new LinkedHashMap<>(); + // Built-ins first so ServiceLoader can override the same id if desired. + register(byId, new ElasticsearchITBackend()); + register(byId, new OpenSearchITBackend()); + for (PersistenceITBackend backend : ServiceLoader.load(PersistenceITBackend.class)) { + register(byId, backend); + } + return byId; + } + + private static void register(Map<String, PersistenceITBackend> byId, PersistenceITBackend backend) { + byId.put(backend.providerId(), backend); + } + + private static PersistenceITBackend instantiate(String fqcn) { + try { + Class<?> clazz = Class.forName(fqcn); + if (!PersistenceITBackend.class.isAssignableFrom(clazz)) { + throw new IllegalArgumentException(fqcn + " does not implement PersistenceITBackend"); + } + return (PersistenceITBackend) clazz.getDeclaredConstructor().newInstance(); + } catch (ReflectiveOperationException e) { + throw new IllegalArgumentException("Failed to instantiate PersistenceITBackend " + fqcn, e); + } + } + + /** Visible for tests / diagnostics. */ + static List<String> knownProviderIds() { + return new ArrayList<>(loadBackendsById().keySet()); + } +} diff --git a/itests/src/test/java/org/apache/unomi/itests/persistence/PersistenceITCapabilities.java b/itests/src/test/java/org/apache/unomi/itests/persistence/PersistenceITCapabilities.java new file mode 100644 index 000000000..707bc4301 --- /dev/null +++ b/itests/src/test/java/org/apache/unomi/itests/persistence/PersistenceITCapabilities.java @@ -0,0 +1,204 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License + */ +package org.apache.unomi.itests.persistence; + +/** + * Backend-agnostic IT capabilities. Describe <em>what</em> the harness and tests may + * rely on (HTTP admin surface, rollover API shape, health probes), not which vendor + * product is installed. + * <p> + * Built-in Elasticsearch / OpenSearch factories set these to match today’s search + * engines; JDBC and other providers typically use {@link #none()}. + */ +public final class PersistenceITCapabilities { + + /** + * HTTP API shape used to inspect or assert index rollover policy wiring. + * Named after the protocol, not a vendor. + */ + public enum IndexRolloverApi { + /** No rollover HTTP API (typical for SQL / embedded stores). */ + NONE, + /** + * Lifecycle-policy HTTP API (paths/settings of the form {@code _ilm}, + * {@code index.lifecycle.*}). + */ + LIFECYCLE, + /** + * State-management-policy HTTP API (paths/settings of the form + * {@code _plugins/_ism}, {@code index_state_management.*}). + */ + STATE_MANAGEMENT; + + public boolean isPresent() { + return this != NONE; + } + } + + /** + * Expected outcome of range queries on flattened / nested document properties. + */ + public enum FlattenedRangeQueryResult { + /** Query succeeds with null or an empty hit list. */ + EMPTY, + /** Query may return matching documents. */ + HITS + } + + private final boolean httpAdminApi; + private final boolean providerNamedHealthProbe; + private final boolean clusterHealthProbe; + private final IndexRolloverApi indexRolloverApi; + private final boolean snapshotRestoreMigration; + private final FlattenedRangeQueryResult flattenedRangeQueryResult; + + private PersistenceITCapabilities(Builder builder) { + this.httpAdminApi = builder.httpAdminApi; + this.providerNamedHealthProbe = builder.providerNamedHealthProbe; + this.clusterHealthProbe = builder.clusterHealthProbe; + this.indexRolloverApi = builder.indexRolloverApi; + this.snapshotRestoreMigration = builder.snapshotRestoreMigration; + this.flattenedRangeQueryResult = builder.flattenedRangeQueryResult; + } + + /** + * Backend exposes HTTP admin APIs for cluster/index/snapshot operations + * (typical of search engines). When false, {@code BaseIT} skips HTTP health + * prep and search helpers refuse to run. + */ + public boolean httpAdminApi() { + return httpAdminApi; + } + + /** + * Health check includes a probe whose name equals the persistence provider id. + */ + public boolean providerNamedHealthProbe() { + return providerNamedHealthProbe; + } + + /** + * Health check includes a cluster-level probe (name {@code cluster}). + */ + public boolean clusterHealthProbe() { + return clusterHealthProbe; + } + + /** + * Rollover policy HTTP API shape, or {@link IndexRolloverApi#NONE}. + */ + public IndexRolloverApi indexRolloverApi() { + return indexRolloverApi; + } + + /** + * Backend can exercise legacy snapshot-restore migration fixtures + * (search-engine snapshot HTTP + restore into current indices). + */ + public boolean snapshotRestoreMigration() { + return snapshotRestoreMigration; + } + + /** + * How range queries on flattened properties behave for assertions. + */ + public FlattenedRangeQueryResult flattenedRangeQueryResult() { + return flattenedRangeQueryResult; + } + + public static Builder builder() { + return new Builder(); + } + + /** No optional capabilities — JDBC, embedded, or incomplete providers. */ + public static PersistenceITCapabilities none() { + return builder().build(); + } + + /** @deprecated use {@link #none()} */ + @Deprecated + public static PersistenceITCapabilities nonSearchBackend() { + return none(); + } + + public static PersistenceITCapabilities elasticsearch() { + return builder() + .httpAdminApi(true) + .providerNamedHealthProbe(true) + .clusterHealthProbe(true) + .indexRolloverApi(IndexRolloverApi.LIFECYCLE) + .snapshotRestoreMigration(true) + .flattenedRangeQueryResult(FlattenedRangeQueryResult.EMPTY) + .build(); + } + + public static PersistenceITCapabilities opensearch() { + return builder() + .httpAdminApi(true) + .providerNamedHealthProbe(true) + .clusterHealthProbe(true) + .indexRolloverApi(IndexRolloverApi.STATE_MANAGEMENT) + .snapshotRestoreMigration(false) + .flattenedRangeQueryResult(FlattenedRangeQueryResult.HITS) + .build(); + } + + public static final class Builder { + private boolean httpAdminApi; + private boolean providerNamedHealthProbe; + private boolean clusterHealthProbe; + private IndexRolloverApi indexRolloverApi = IndexRolloverApi.NONE; + private boolean snapshotRestoreMigration; + private FlattenedRangeQueryResult flattenedRangeQueryResult = FlattenedRangeQueryResult.EMPTY; + + public Builder httpAdminApi(boolean httpAdminApi) { + this.httpAdminApi = httpAdminApi; + return this; + } + + public Builder providerNamedHealthProbe(boolean providerNamedHealthProbe) { + this.providerNamedHealthProbe = providerNamedHealthProbe; + return this; + } + + public Builder clusterHealthProbe(boolean clusterHealthProbe) { + this.clusterHealthProbe = clusterHealthProbe; + return this; + } + + public Builder indexRolloverApi(IndexRolloverApi indexRolloverApi) { + this.indexRolloverApi = indexRolloverApi != null ? indexRolloverApi : IndexRolloverApi.NONE; + return this; + } + + public Builder snapshotRestoreMigration(boolean snapshotRestoreMigration) { + this.snapshotRestoreMigration = snapshotRestoreMigration; + return this; + } + + public Builder flattenedRangeQueryResult(FlattenedRangeQueryResult flattenedRangeQueryResult) { + this.flattenedRangeQueryResult = flattenedRangeQueryResult != null + ? flattenedRangeQueryResult + : FlattenedRangeQueryResult.EMPTY; + return this; + } + + public PersistenceITCapabilities build() { + return new PersistenceITCapabilities(this); + } + } +} diff --git a/itests/src/test/java/org/apache/unomi/itests/persistence/SearchBackendIT.java b/itests/src/test/java/org/apache/unomi/itests/persistence/SearchBackendIT.java new file mode 100644 index 000000000..9f8f81820 --- /dev/null +++ b/itests/src/test/java/org/apache/unomi/itests/persistence/SearchBackendIT.java @@ -0,0 +1,29 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License + */ +package org.apache.unomi.itests.persistence; + +/** + * Marker category for ITs whose <em>primary</em> assertions need HTTP admin APIs + * (snapshot restore, index rollover policy HTTP). Prefer keeping these classes in + * {@link org.apache.unomi.itests.CorePersistenceITs} / {@link org.apache.unomi.itests.AllITs} + * and gating with {@link org.junit.Assume} on {@link PersistenceITCapabilities} so unsupported + * backends skip rather than losing suite membership. + * <p> + * Optional Failsafe {@code excludedGroups} may still list this category for specialized cells. + */ +public interface SearchBackendIT { +} diff --git a/itests/src/test/resources/META-INF/services/org.apache.unomi.itests.persistence.PersistenceITBackend b/itests/src/test/resources/META-INF/services/org.apache.unomi.itests.persistence.PersistenceITBackend new file mode 100644 index 000000000..46eb7e9d3 --- /dev/null +++ b/itests/src/test/resources/META-INF/services/org.apache.unomi.itests.persistence.PersistenceITBackend @@ -0,0 +1,2 @@ +org.apache.unomi.itests.persistence.ElasticsearchITBackend +org.apache.unomi.itests.persistence.OpenSearchITBackend
