Thanks all. I will look into these
From: Matt Burgess <[email protected]> Sent: July 16, 2018 3:42 PM To: [email protected] Subject: Re: Send data to ElasticSearch There’s a PutElasticsearchHttpRecord that should give you the best of both worlds, the Record API to convert data types and using the REST API. On Jul 16, 2018, at 2:59 PM, Mike Thomsen <[email protected]<mailto:[email protected]>> wrote: With PutElasticsearchHttp, you'll have to define the field in advance as a date one because it's not a record-aware processor (the Record API has data types that can solve this problem). Once you've defined your date fields, an ISO8601 date string will suffice. If you have more than one index you'll need to write to, and the date field names are the same, I'd recommend writing an ElasticSearch index template to apply the rule to all indexes that match the supplied name pattern. On Mon, Jul 16, 2018 at 2:36 PM Jean-Sebastien Vachon <[email protected]<mailto:[email protected]>> wrote: Hi all, I’m sending the result of my flow to ElasticSearch for indexing but ES fails to recognize my timestamp field for a date based field. What is the recommended approach to fix this? I thought about adding some steps to my flow that would check if a mapping exists for my index and the send the mappings to ES if required. In case where the Mapping do not exists, ES will return something like: { "error": { "root_cause": [ { "type": "index_not_found_exception", "reason": "no such index", "resource.type": "index_or_alias", "resource.id<http://resource.id>": "my_index", "index_uuid": "_na_", "index": "my_index" } ], "type": "index_not_found_exception", "reason": "no such index", "resource.type": "index_or_alias", "resource.id<http://resource.id>": "my_index", "index_uuid": "_na_", "index": "my_index" }, "status": 404 } So, If I fetch the response and look out for “status:404”, I would know whether or not I need to create the mappings. In this case, I would call a PutElasticSearchHttp with the proper mappings and everything should be fine. However, this seems like a lot of trouble for an operation that will most likely failed only once. Are there any other ways of dealing with this? Thanks -- Jean-Sébastien Vachon [email protected]<mailto:[email protected]> [email protected]<mailto:[email protected]> www.brizodata.com<http://www.brizodata.com/>
