fgerlits commented on code in PR #1349:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1349#discussion_r907494696
##########
bstrp_functions.sh:
##########
@@ -386,6 +386,7 @@ show_supported_features() {
echo "AB. Kubernetes Support .........$(print_feature_status
KUBERNETES_ENABLED)"
echo "AC. Google Cloud Support .......$(print_feature_status GCP_ENABLED)"
echo "AD. ProcFs Support .............$(print_feature_status PROCFS_ENABLED)"
+ echo "AE. Elasticsearch Support ......$(print_feature_status
ELASTIC_ENABLED)"
Review Comment:
lines 412 and 467 need to be updated
##########
docker/test/integration/features/elasticsearch.feature:
##########
@@ -0,0 +1,93 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+@no-ci # Elasticsearch container requires more RAM than what the CI
environment has
+Feature: Managing documents on Elasticsearch with PostElasticsearch
+
+ Background:
+ Given the content of "/tmp/output" is monitored
+
+ Scenario: MiNiFi instance indexes a document on Elasticsearch using Basic
Authentication
+ Given an Elasticsearch server is set up and running
+ And a GetFile processor with the "Input Directory" property set to
"/tmp/input"
+ And a file with the content "{ "field1" : "value1" }" is present in
"/tmp/input"
+ And a PostElasticsearch processor
+ And the "Index" property of the PostElasticsearch processor is set to
"my_index"
+ And the "Identifier" property of the PostElasticsearch processor is set to
"my_id"
+ And the "Action" property of the PostElasticsearch processor is set to
"index"
+ And a SSL context service is set up for PostElasticsearch and Elasticsearch
+ And an ElasticsearchCredentialsService is set up for PostElasticsearch
with Basic Authentication
+ And a PutFile processor with the "Directory" property set to "/tmp/output"
+ And the "success" relationship of the GetFile processor is connected to
the PostElasticsearch
+ And the "success" relationship of the PostElasticsearch processor is
connected to the PutFile
+
+ When both instances start up
+ Then a flowfile with the content "{ "field1" : "value1" }" is placed in
the monitored directory in less than 20 seconds
+ And Elasticsearch has a document with "my_id" in "my_index" that has
"value1" set in "field1"
+
+ Scenario: MiNiFi instance creates a document on Elasticsearch using API Key
authentication
Review Comment:
Are these two scenarios identical except for `Basic Authentication` /
`ApiKey`? If so, then they could be merged into a single Scenario Outline with
two Examples.
##########
docker/DockerVerify.sh:
##########
@@ -67,7 +67,7 @@ TEST_DIRECTORY="${docker_dir}/test/integration"
export TEST_DIRECTORY
# Add --no-logcapture to see logs interleaved with the test output
-BEHAVE_OPTS=(-f pretty --logging-level INFO --logging-clear-handlers)
+BEHAVE_OPTS=(-f pretty --logging-level INFO --logging-clear-handlers --tags
~@no-ci)
Review Comment:
Can we have a command-line argument or environment variable to control this?
I would like `make docker-verify` to include these tests on my computer, but
skip them in CI jobs.
##########
docker/test/integration/steps/steps.py:
##########
@@ -410,6 +411,62 @@ def step_impl(context):
context.test.acquire_container("fake-gcs-server", "fake-gcs-server")
+# elasticsearch
+@given('an Elasticsearch server is set up and running')
+@given('an Elasticsearch server is set up and a single document is present
with "preloaded_id" in "my_index"')
+@given('an Elasticsearch server is set up and a single document is present
with "preloaded_id" in "my_index" with "value1" in "field1"')
+def step_impl(context):
+ context.test.start_elasticsearch()
+ context.test.create_doc_elasticsearch("elasticsearch", "my_index",
"preloaded_id")
+
+
+# opensearch
+@given('an Opensearch server is set up and running')
+@given('an Opensearch server is set up and a single document is present with
"preloaded_id" in "my_index"')
+@given('an Opensearch server is set up and a single document is present with
"preloaded_id" in "my_index" with "value1" in "field1"')
+def step_impl(context):
+ context.test.start_opensearch()
+ context.test.add_elastic_user_to_opensearch("opensearch")
+ context.test.create_doc_elasticsearch("opensearch", "my_index",
"preloaded_id")
+
+
+@given(u'a SSL context service is set up for PostElasticsearch and
Elasticsearch')
+def step_impl(context):
+ minifi_crt_file = '/tmp/resources/elasticsearch/minifi_client.crt'
+ minifi_key_file = '/tmp/resources/elasticsearch/minifi_client.key'
+ root_ca_crt_file = '/tmp/resources/elasticsearch/root_ca.crt'
+ ssl_context_service = SSLContextService(cert=minifi_crt_file,
ca_cert=root_ca_crt_file, key=minifi_key_file)
+ put_elasticsearch_json = context.test.get_node_by_name("PostElasticsearch")
+ put_elasticsearch_json.controller_services.append(ssl_context_service)
+ put_elasticsearch_json.set_property("SSL Context Service",
ssl_context_service.name)
Review Comment:
very minor, but why are the variables containing the `PostElasticsearch`
processor called `put_elastic...` instead of `post_elastic...` everywhere in
this file?
##########
docker/test/integration/features/opensearch.feature:
##########
@@ -0,0 +1,97 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+@no-ci # Opensearch container requires more RAM than what the CI environment
has
+Feature: PostElasticsearch works on Opensearch (Opensearch doesnt support API
Keys)
+
+ Background:
+ Given the content of "/tmp/output" is monitored
+
+ Scenario: MiNiFi instance indexes a document on Opensearch using Basic
Authentication
+ Given an Opensearch server is set up and running
+ And a GetFile processor with the "Input Directory" property set to
"/tmp/input"
+ And a file with the content "{ "field1" : "value1" }" is present in
"/tmp/input"
+ And a PostElasticsearch processor
+ And the "Hosts" property of the PostElasticsearch processor is set to
"https://opensearch:9200";
+ And the "Index" property of the PostElasticsearch processor is set to
"my_index"
+ And the "Identifier" property of the PostElasticsearch processor is set to
"my_id"
+ And the "Action" property of the PostElasticsearch processor is set to
"index"
+ And a SSL context service is set up for PostElasticsearch and Opensearch
+ And an ElasticsearchCredentialsService is set up for PostElasticsearch
with Basic Authentication
+ And a PutFile processor with the "Directory" property set to "/tmp/output"
+ And the "success" relationship of the GetFile processor is connected to
the PostElasticsearch
+ And the "success" relationship of the PostElasticsearch processor is
connected to the PutFile
+
+ When both instances start up
+ Then a flowfile with the content "{ "field1" : "value1" }" is placed in
the monitored directory in less than 20 seconds
+ And Opensearch has a document with "my_id" in "my_index" that has "value1"
set in "field1"
+
+ Scenario: MiNiFi instance creates a document on Opensearch using Basic
Authentication
Review Comment:
Here, too, it looks like the only difference between the first two Scenarios
is the value of the Action property, so they could be merged into a single
Scenario Outline.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
