Jon,

In 1.6, we added a new client service for ES. There's only one processor
that uses it for now, JsonQueryElasticSearch, but I have more under way
that I hope will make it into 1.7. If you take a look at the existing
version of the client service implementation that uses the official
ElasticSearch APIs. You might be able to quickly get this up and running
with a new implementation that uses the AWS SDK instead of the official API.

FYI, JsonQueryElasticSearch is designed so you can build a query in Kibana
and just drop it into NiFi and go. So any querying other than scrolls
should work out of the box if you create a proper implementation of
ElasticSearchClientService that uses the Amazon code.

On Mon, Apr 9, 2018 at 8:29 PM, Otto Fowler <ottobackwa...@gmail.com> wrote:

> The aws java sdk has a purpose built ElasticSearchClient class.
> The way to do this and be consistent would be to add a new nifi-aws
> processor for the ES client,
> as there is for s3 and dynamoDB etc.
>
> https://github.com/apache/nifi/pull/2588 is my outstanding PR for
> HttpInvoke for AWS Gateway Web APIs.
> A similar approach being having version of the ElasticSearchHttp but in the
> aws nar.
>
>
>
> On April 9, 2018 at 19:11:45, Jon Logan (jmlo...@buffalo.edu) wrote:
>
> Hi All,
>
> We are looking to use the built-in Nifi Elasticsearch Http processors with
> signed AWS ES requests. I tried to extend them to do so, but ran into
> issues with things being in-extensible (private-static-final in some
> cases), and was wondering if this would be something that would be
> considered to be merged into the baseline?
>
> There's two main ways I saw doing it -- either modifying the existing ES
> code to allow for it, or making new AWS-specific extended adapters to do
> it. In the former, it might require dependencies between the ES code and
> the AWS code for optional CredentialProviders, etc., and am not sure how
> isolated you all try to keep things.
>
> In either case, AWS essentially adds an HTTP-header signature to
> authenticate the request against your ES instance. The easiest path to do
> this seems to be writing a bridge between the ES OkHttpClient requests and
> the AWS requests to generate the signature correctly.
>
>
> Just wanted to get some feedback to be sure this wouldn't be a waste of
> time.
>
>
> Thanks!
> Jon
>

Reply via email to