ChrisSamo632 commented on a change in pull request #4693:
URL: https://github.com/apache/nifi/pull/4693#discussion_r724354340
##########
File path:
nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-client-service/src/main/java/org/apache/nifi/elasticsearch/ElasticSearchClientServiceImpl.java
##########
@@ -182,8 +181,8 @@ private Response runQuery(final String endpoint, final
String query, final Strin
final HttpEntity queryEntity = new NStringEntity(query,
ContentType.APPLICATION_JSON);
try {
- return client.performRequest("POST", sb.toString(),
requestParameters != null ? requestParameters : Collections.emptyMap(),
queryEntity);
- } catch (final Exception e) {
+ return performRequest("POST", sb.toString(), requestParameters,
queryEntity);
Review comment:
Not sure what you want to be `final` here? I've applied `final` to
everywhere I can see it may be usable within the class (there were a couple of
things I'd missed previously, but this line isn't assigning any variables, so
there's nothing to make `final`)?
##########
File path:
nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-client-service/src/main/java/org/apache/nifi/elasticsearch/ElasticSearchClientServiceImpl.java
##########
@@ -359,9 +358,21 @@ public DeleteOperationResponse deleteByQuery(final String
query, final String in
return new
DeleteOperationResponse(watch.getDuration(TimeUnit.MILLISECONDS));
}
+
+ public UpdateOperationResponse updateByQuery(final String query, final
String index, final String type, final Map<String, String> requestParameters) {
+ long start = System.currentTimeMillis();
+ Response response = runQuery("_update_by_query", query, index, type,
requestParameters);
+ long end = System.currentTimeMillis();
Review comment:
Think this was an existing thing, I generally don't like aligning things
in thiw way either, so happy to change 🤷
##########
File path:
nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-restapi-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearchRecord.java
##########
@@ -414,8 +434,8 @@ private void removeBadRecordFlowFiles(List<FlowFile> bad,
ProcessSession session
bad.clear();
}
- private FlowFile indexDocuments(BulkOperation bundle, ProcessSession
session, FlowFile input) throws Exception {
- IndexOperationResponse response =
clientService.bulk(bundle.getOperationList());
+ private FlowFile indexDocuments(BulkOperation bundle, ProcessContext
context, ProcessSession session, FlowFile input) throws Exception {
+ IndexOperationResponse response =
clientService.bulk(bundle.getOperationList(), getUrlQueryParameters(context,
input));
Review comment:
`final`s, `final`s everywhere 😉
##########
File path:
nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-restapi-processors/src/main/java/org/apache/nifi/processors/elasticsearch/SearchElasticsearch.java
##########
@@ -65,6 +66,11 @@
"Search After/Point in Time queries must include a valid \"sort\"
field. The processor will retrieve multiple pages of results " +
"until either no more results are available or the Pagination Keep
Alive expiration is reached, after which the query will " +
"restart with the first page of results being retrieved.")
+@DynamicProperty(
+ name = "A URL query parameter",
+ value = "The value to set it to",
+ expressionLanguageScope = ExpressionLanguageScope.FLOWFILE_ATTRIBUTES,
+ description = "Adds the specified property name/value as a query
parameter in the Elasticsearch URL used for processing")
Review comment:
I'm confused, which other comments are you referring to (I can't make
this `final` 😛)?
##########
File path:
nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-restapi-processors/src/main/java/org/apache/nifi/processors/elasticsearch/UpdateByQueryElasticsearch.java
##########
@@ -0,0 +1,64 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.elasticsearch;
+
+import org.apache.nifi.annotation.behavior.DynamicProperty;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.elasticsearch.ElasticSearchClientService;
+import org.apache.nifi.elasticsearch.OperationResponse;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+
+import java.util.Map;
+
+@WritesAttributes({
+ @WritesAttribute(attribute = "elasticsearch.update.took", description
= "The amount of time that it took to complete the update operation in ms."),
+ @WritesAttribute(attribute = "elasticsearch.update.error", description
= "The error message provided by Elasticsearch if there is an error running the
update.")
+})
+@InputRequirement(InputRequirement.Requirement.INPUT_ALLOWED)
+@Tags({ "elastic", "elasticsearch", "update", "query"})
+@CapabilityDescription("Update documents in an Elasticsearch index using a
query. The query can be loaded from a flowfile body " +
+ "or from the Query parameter.")
+@DynamicProperty(
+ name = "A URL query parameter",
+ value = "The value to set it to",
+ expressionLanguageScope = ExpressionLanguageScope.FLOWFILE_ATTRIBUTES,
Review comment:
Which other comments?
##########
File path:
nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-restapi-processors/src/test/groovy/org/apache/nifi/processors/elasticsearch/AbstractByQueryElasticsearchTest.groovy
##########
@@ -0,0 +1,218 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License") you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.elasticsearch
+
+import org.apache.nifi.util.MockFlowFile
+import org.apache.nifi.util.TestRunner
+import org.junit.Assert
+import org.junit.Test
+
+abstract class AbstractByQueryElasticsearchTest {
Review comment:
While it should be fairly straight forward to convert them, I think that
should be part of a larger initiative if/when decided upon. Also, we couldn't
convert just this one class, we'd need to convert all of the tests (pretty
much) in this package.
Also, currently, all the tests in the package are Groovy and not Java, I
think moving just a sub-set of the tests would lead to more confusion rather
than solving anything.
Maybe we should raise a follow-on Jira ticket to suggest the move and see
what people think?
##########
File path:
nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-restapi-processors/src/main/java/org/apache/nifi/processors/elasticsearch/SearchElasticsearch.java
##########
@@ -65,6 +66,11 @@
"Search After/Point in Time queries must include a valid \"sort\"
field. The processor will retrieve multiple pages of results " +
"until either no more results are available or the Pagination Keep
Alive expiration is reached, after which the query will " +
"restart with the first page of results being retrieved.")
+@DynamicProperty(
+ name = "A URL query parameter",
+ value = "The value to set it to",
+ expressionLanguageScope = ExpressionLanguageScope.FLOWFILE_ATTRIBUTES,
+ description = "Adds the specified property name/value as a query
parameter in the Elasticsearch URL used for processing")
Review comment:
Ah wait, I'd missed a load of comments (GitHub hides a lot by default
where there are many comments... I'll look through the rest shortly)
##########
File path:
nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-restapi-processors/src/main/java/org/apache/nifi/processors/elasticsearch/SearchElasticsearch.java
##########
@@ -65,6 +66,11 @@
"Search After/Point in Time queries must include a valid \"sort\"
field. The processor will retrieve multiple pages of results " +
"until either no more results are available or the Pagination Keep
Alive expiration is reached, after which the query will " +
"restart with the first page of results being retrieved.")
+@DynamicProperty(
+ name = "A URL query parameter",
+ value = "The value to set it to",
+ expressionLanguageScope = ExpressionLanguageScope.FLOWFILE_ATTRIBUTES,
+ description = "Adds the specified property name/value as a query
parameter in the Elasticsearch URL used for processing")
Review comment:
Found the relevant comments, addressed
##########
File path:
nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-restapi-processors/src/test/groovy/org/apache/nifi/processors/elasticsearch/AbstractByQueryElasticsearchTest.groovy
##########
@@ -0,0 +1,218 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License") you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.elasticsearch
+
+import org.apache.nifi.util.MockFlowFile
+import org.apache.nifi.util.TestRunner
+import org.junit.Assert
+import org.junit.Test
+
+abstract class AbstractByQueryElasticsearchTest {
Review comment:
[NIFI-9290](https://issues.apache.org/jira/browse/NIFI-9290) raised to
cover this consideration
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
