The GitHub Actions job "Java CI with Maven" on stormcrawler.git/main has succeeded. Run started by GitHub user jnioche (triggered by jnioche).
Head commit for run: 1b56770c5306b0027cc92a763dfc1ea14a52fcdc / Davide Polato <[email protected]> #2212 move the opensearch-java implementation into external/opensearch (#2242) * accept null metadata values in the opensearch-java bolts Metadata.setValue(key, null) stores a null element, and addValue keeps it next to the new value. StatusUpdaterBolt copied the values of every metadata key with List.of, and IndexerBolt the values of the keys with more than one value. List.of throws a NullPointerException on a null element; the bolts caught it and failed the tuple. Both bolts now copy the values with Arrays.asList, and the null is written as a JSON null inside the array, as the legacy module does. Signed-off-by: Davide Polato <[email protected]> * replace the legacy opensearch sources with the opensearch-java ones The files of external/opensearch/src that also exist in external/opensearch-java take the content of their opensearch-java counterparts in place, so git blame keeps the lines that did not change. The files that exist only in external/opensearch-java are moved. The following commits update the module's POM, configuration and README. Signed-off-by: Davide Polato <[email protected]> * build external/opensearch with the opensearch-java client The module depends on opensearch-java 3.10.0 instead of the high-level REST client and the sniffer, and its tests run against OpenSearch 3.5.0. The artifactId stays stormcrawler-opensearch. external/opensearch-java leaves the reactor; its README is merged into the one of external/opensearch in a later commit. Signed-off-by: Davide Polato <[email protected]> * drop the sniff and responseBufferSize settings The opensearch-java client has no sniffer and no per-request response buffer setting. The keys are removed from the module's opensearch-conf.yaml, from the one of the archetype and from the configuration reference. When they are set, OpenSearchConnection still logs a WARN, which now points to external/opensearch/README.md. The configuration reference and the Javadoc of DelegateRefresher no longer mention a second OpenSearch module. Signed-off-by: Davide Polato <[email protected]> * merge the opensearch-java README into external/opensearch The README of external/opensearch describes the client and AsyncBulkProcessor, and keeps the archetype and dashboard instructions. A new section lists what changes for users of earlier versions: the configuration keys, the behaviour differences and the types that custom code has to use. Signed-off-by: Davide Polato <[email protected]> * port the legacy tests for credentials and WaitAckCache shutdown The legacy module tested that a running client sends the credentials to a configured address and not to another one, that credentials need both a user and a password, that host and scheme are matched without regard to case, and that WaitAckCache.shutdown() fails the tuples still waiting. The opensearch-java tests only checked how the credential scopes are built. The tests are adapted to HttpClient 5 and the opensearch-java transport. Two legacy tests are not carried over: the sniffer scheme test, since there is no sniffer, and the test for a credentials lookup without an origin, since the transport's preemptive Basic scheme looks the credentials up with the host of the request. Signed-off-by: Davide Polato <[email protected]> * keep the connection pool limits when TLS validation is disabled With opensearch.disable.tls.validation set, OpenSearchConnection replaces the connection manager of the transport to install a trust-all TLS strategy. The replacement had no pool limits, so HttpClient 5 allowed 5 connections per node and 25 in total, instead of the 10 and 30 that the transport sets. The replacement now uses the transport's limits. The new test sends 12 requests to a local server that never answers and checks that 10 connections are open. Signed-off-by: Davide Polato <[email protected]> * upgrade server Signed-off-by: Davide Polato <[email protected]> * turn off the bulk count limit for a negative bulkActions The legacy BulkProcessor treats bulkActions -1 as no limit on the number of operations. AsyncBulkProcessor used the value as the capacity of its buffer, so -1 made the bolt fail at startup. A negative value now turns off the count limit, while the size limit and the flush interval still apply. 0 keeps sending each operation in its own bulk, as in both modules. Signed-off-by: Davide Polato <[email protected]> * warn about the global sniff and responseBufferSize keys OpenSearchConnection logged a WARN only for the per-type keys, so a global opensearch.sniff or opensearch.responseBufferSize was ignored without a warning. The WARN now covers both forms and names the key that is set. Signed-off-by: Davide Polato <[email protected]> * run OpenSearch 3.9.0 in the archetype's docker-compose.yml The module tests run against OpenSearch 3.9.0, while the archetype's compose file still started 2.19.4 for OpenSearch and Dashboards. AbstractOpenSearchTest now also falls back to 3.9.0 when the opensearch-version property is not set, as in a run from an IDE. Signed-off-by: Davide Polato <[email protected]> * fail at startup when a mapping resource is missing IndexCreation logged a WARN when the <boltType>.mapping resource was not on the classpath, and the component started without its index or template. It now looks the resource up with Resources.getResource before the try, which throws IllegalArgumentException naming the resource, so the bolt, spout or metrics consumer fails in prepare() or open(). The README no longer lists the WARN among the changes in behaviour. Signed-off-by: Davide Polato <[email protected]> * minor enhancement: log why an index or template was not created createIndex and createTemplate returned false when OpenSearch answered the creation with an error status, and checkOrCreateIndex only logged that the index was not created. They now log a WARN with the status and the response body, as the legacy module did with the exception of the high-level client. The comments that said they only log exceptions are removed. IndexCreationTest checks both paths with a mocked 400. * drop or shorten comments --------- Signed-off-by: Davide Polato <[email protected]> Report URL: https://github.com/apache/stormcrawler/actions/runs/39735916306 With regards, GitHub Actions via GitBox
