[
https://issues.apache.org/jira/browse/NIFI-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16482969#comment-16482969
]
ASF GitHub Bot commented on NIFI-5051:
--------------------------------------
Github user mattyb149 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2615#discussion_r189696292
--- Diff:
nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-client-service/src/main/java/org/apache/nifi/elasticsearch/ElasticSearchLookupService.java
---
@@ -0,0 +1,253 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.elasticsearch;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.lookup.LookupFailureException;
+import org.apache.nifi.lookup.LookupService;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.schema.access.SchemaNotFoundException;
+import org.apache.nifi.serialization.SchemaRegistryService;
+import org.apache.nifi.serialization.SimpleRecordSchema;
+import org.apache.nifi.serialization.record.MapRecord;
+import org.apache.nifi.serialization.record.Record;
+import org.apache.nifi.serialization.record.RecordField;
+import org.apache.nifi.serialization.record.RecordFieldType;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.serialization.record.type.RecordDataType;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+public class ElasticSearchLookupService extends SchemaRegistryService
implements LookupService {
+ public static final PropertyDescriptor CLIENT_SERVICE = new
PropertyDescriptor.Builder()
+ .name("el-rest-client-service")
+ .displayName("Client Service")
+ .description("An ElasticSearch client service to use for running
queries.")
+ .identifiesControllerService(ElasticSearchClientService.class)
+ .required(true)
+ .build();
+ public static final PropertyDescriptor INDEX = new
PropertyDescriptor.Builder()
+ .name("el-lookup-index")
+ .displayName("Index")
+ .description("The name of the index to read from")
+ .required(true)
+
.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
+ .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+ .build();
+
+ public static final PropertyDescriptor TYPE = new
PropertyDescriptor.Builder()
+ .name("el-lookup-type")
+ .displayName("Type")
+ .description("The type of this document (used by Elasticsearch for
indexing and searching)")
+ .required(false)
+
.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
+ .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+ .build();
+
+
+ public static final PropertyDescriptor RECORD_SCHEMA_NAME = new
PropertyDescriptor.Builder()
+ .name("el-lookup-record-schema-name")
+ .displayName("Record Schema Name")
+ .description("If specified, the value will be used to lookup a
schema in the configured schema registry.")
+ .required(false)
+ .addValidator(Validator.VALID)
+ .build();
+
+ private ElasticSearchClientService clientService;
+
+ private String index;
+ private String type;
+ private ObjectMapper mapper;
+
+ @OnEnabled
+ public void onEnabled(final ConfigurationContext context) throws
InitializationException {
+ clientService =
context.getProperty(CLIENT_SERVICE).asControllerService(ElasticSearchClientService.class);
+ index = context.getProperty(INDEX).getValue();
+ type = context.getProperty(TYPE).getValue();
+ mapper = new ObjectMapper();
+ }
+
+ @Override
+ protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
+ List<PropertyDescriptor> _desc = new ArrayList<>();
+ _desc.addAll(super.getSupportedPropertyDescriptors());
+ _desc.add(CLIENT_SERVICE);
+ _desc.add(INDEX);
+ _desc.add(TYPE);
+ _desc.add(RECORD_SCHEMA_NAME);
+
+ return Collections.unmodifiableList(_desc);
+ }
+
+ @Override
+ public Optional lookup(Map coordinates) throws LookupFailureException {
--- End diff --
I kinda thought this LookupService would behave a bit like the Mongo one,
where you could give it multiple keys and it would do the query based on that
(the fields and the values for each record). This one seems a bit awkward to
me, as the user would have to build up their own query field in each record,
putting the value they want to match inside a JSON query body.
Is there a different use case here, or could/should we make it more
consistent with the other "NoSQL" lookup service(s)? We'd have to generate the
query body but that shouldn't be too hard. Also you'd only be able to query
top-level fields for lookup, but that seems like it would cover most use cases.
If there is a way to specify a nested field for lookup (such as a qualified
name with period delimiters), we could do that (although we'd likely have to
use a "nested" operator in the generated query), seems like a good (but
separate) improvement. Thoughts?
> Create a LookupService that uses ElasticSearch
> ----------------------------------------------
>
> Key: NIFI-5051
> URL: https://issues.apache.org/jira/browse/NIFI-5051
> Project: Apache NiFi
> Issue Type: New Feature
> Reporter: Mike Thomsen
> Assignee: Mike Thomsen
> Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
