Eli, The best starting point we have is the nifi-elasticsearch-restapi-bundle module and its sub-modules. That uses a client service for most of the connection management. It's based on the official Elastic client APIs. My recommendation would be to clone that module and start creating a version agnostic OpenSearch bundle.
Mike On Mon, Nov 15, 2021 at 1:00 PM Chris Sampson <[email protected]> wrote: > > Note that I'd started to consider this in some recent updates to the NiFi > Elasticsearch REST Processors[1]. It looks like my comments/the actual > version in this file may have got a little out of step with one another, I > think I intended to use 7.13.4 in this pom.xml as the high-level REST > Clients from Elastic are still Apache 2.0 licensed, it's the low-level > client that's no longer Apache 2.0 (I may look into this further if I get > chance, but that's irrelevant for any OpenSearch work). > > OpenSearch is not something I've tested against nor plan to make changes > for in any ongoing work, but I'd imagine a sensible place to start/consider > would be to look at providing a new implementation of the > ElasticSearchClientService[2] that uses the same API (so far as the NiFi > processors are concerned). This would then allow users to select an > appropriate implementation (e.g. the existing > ElasticSearchClientServiceImpl[3] or a new OpenSearch equivalent) for use > in any of the existing Elasticsearch REST Processors [4]. > > I'd probably suggest not updating the other Elasticsearch processors: > * nifi-elasticsearch-5-processors have largely been replaced and are only > intended for use with the now very old Elasticsearch 5.x branch > * nifi-elasticsearch-processors all have > equivalent nifi-elasticsearch-restapi-processors processors that use the > above Service API (except for the FetchElasticsearch processors, for which > we should probably raise a Jira ticket to create a newer equivalent, thus > allowing potential deprecation of all processors other than the restapi set) > > > [1] > https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-client-service/pom.xml#L142 > [2] > https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-client-service-api/src/main/java/org/apache/nifi/elasticsearch/ElasticSearchClientService.java > [3] > https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-client-service/src/main/java/org/apache/nifi/elasticsearch/ElasticSearchClientServiceImpl.java > [4] > https://github.com/apache/nifi/tree/main/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-restapi-processors/src/main/java/org/apache/nifi/processors/elasticsearch > > --- > *Chris Sampson* > IT Consultant > [email protected] > > > On Mon, 15 Nov 2021 at 17:37, Joe Witt <[email protected]> wrote: > > > Eli, > > > > I've added you and your two cc'd colleagues as bcc so you see this. > > Please subscribe to the list [1] so you'll see the replies and so that > > your emails won't require moderation. cc's and bcc's and such will not > > work for replies/etc.. on these lists. > > > > Ultimately we'd just need someone in the community to create the > > appropriate Opensearch equivalent clients to the existing > > Elasticsearch set. Presuming the License of the client and any > > transitive deps is fair game we can include it or make it available as > > a usable nar supported by the NiFi community. And then we need > > someone to review and ultimately someone with commit privileges to > > merge. > > > > [1] https://nifi.apache.org/mailing_lists.html > > > > Thanks > > > > On Mon, Nov 15, 2021 at 10:22 AM Fisher, Eli <[email protected]> > > wrote: > > > > > > Hi Apache NiFi team, > > > > > > My name is Eli Fisher. I work at Amazon on the OpenSearch project< > > http://opensearch.org/>. We have had a couple people in the community ask > > about adding NiFi support, and so I wanted to reach out to learn more about > > what the process involved would be and to understand if you also seeing of > > related issues/requests. OpenSearch is wire compatible with OSS > > Elasticsearch 7.10.2, and so we have people pointing people to use the > > existing Elasticsearch integration. However, in the fullness of time, these > > projects will likely diverge so we wanted to reach out to learn about the > > process of contributing an integration for OpenSearch. I’ve cc’d two of my > > colleagues who also work on OpenSearch with me that are interested in > > learning about the process. > > > > > > Thank you for your time, > > > Eli > >
