Hi Sandeep
The error is :
{"error":{"root_cause":[{"type":"action_request_validation_exception","reason":"Validation
Failed: 1: no requests
added;"}],"type":"action_request_validation_exception","reason":"Validation
Failed: 1: no requests added;"},"status":400}
I don't find how to rewrite rules.
Best
Rabii
Le ven. 5 oct. 2018 à 18:53, Sandeep Moré <moresand...@gmail.com> a écrit :
> When you say it does not work what error do you get ? do you see the
> rewrite rules for header getting triggered in the debug logs ?
> Also, what is the exception you get on the elasticsearch side.
>
> Best,
> Sandeep
>
> On Fri, Oct 5, 2018 at 11:23 AM rabii lamriq <lam...@gmail.com> wrote:
>
>> Hi All,
>>
>> I tried to change the content-type of the http header in the
>> ElasticsearchDispatcher java class, but doesn't work :(
>>
>> Anyone can give help where we should change the content-type for make
>> "application/x-ndjson" in class of ElasticsearchDispatcher (attached)
>>
>> Regards
>> Rabii
>>
>> Le mar. 25 sept. 2018 à 16:20, rabii lamriq <lam...@gmail.com> a écrit :
>>
>>> I am not understand how to apply this for content-type.
>>>
>>> Le mar. 25 sept. 2018 à 15:18, Sandeep Moré <moresand...@gmail.com> a
>>> écrit :
>>>
>>>> This is an example in Webhdfs service def.
>>>> https://github.com/apache/knox/blob/6e7266ad3649e1804ee869afed04f79a71e48b7d/gateway-service-definitions/src/main/resources/services/webhdfs/2.4.0/rewrite.xml#L64
>>>>
>>>>
>>>>
>>>> On Tue, Sep 25, 2018 at 5:37 AM rabii lamriq <lam...@gmail.com> wrote:
>>>>
>>>>> Hi Sandeep,
>>>>>
>>>>> Yes, I know that is possible, but I don't know how to rewrite the rule
>>>>> to access to the header and change the Content-type, I can't find any
>>>>> thing
>>>>> related to this customization.
>>>>>
>>>>> Regards
>>>>> Rabii
>>>>>
>>>>> Le lun. 24 sept. 2018 à 17:19, Sandeep Moré <moresand...@gmail.com> a
>>>>> écrit :
>>>>>
>>>>>> Yes, you can use the rewrite rules to update Headers, I don't
>>>>>> remember exactly which service uses it but there should be few services
>>>>>> that rewrite headers.
>>>>>>
>>>>>> Best,
>>>>>> Sandeep
>>>>>>
>>>>>> On Mon, Sep 24, 2018 at 10:45 AM rabii lamriq <lam...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>>
>>>>>>> Rewrite rules for knox (link
>>>>>>> <https://cwiki.apache.org/confluence/display/KNOX/2017/08/14/Understanding+Rewrite+Rules+for+Apache+Knox#UnderstandingRewriteRulesforApacheKnox-JSONURLRewriteFIlter>)
>>>>>>> can help me to change header? is it possible?
>>>>>>>
>>>>>>> Le lun. 24 sept. 2018 à 16:43, rabii lamriq <lam...@gmail.com> a
>>>>>>> écrit :
>>>>>>>
>>>>>>>> Hello
>>>>>>>>
>>>>>>>> Any help please??
>>>>>>>>
>>>>>>>> Le lun. 24 sept. 2018 à 09:31, rabii lamriq <lam...@gmail.com> a
>>>>>>>> écrit :
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> thanks every One, for your feedbacks,
>>>>>>>>>
>>>>>>>>> Yes, ES accept both application/json and application/x-ndjson,
>>>>>>>>> there is no problem in ES ni in Grafana, when i test my query with
>>>>>>>>> curl to
>>>>>>>>> send POST to ES, it work well ether json or x-ndjson.
>>>>>>>>>
>>>>>>>>> However, through Knox Gateway, it work only for x-ndjson, I don't
>>>>>>>>> know why. so, since Grafana send their POST with only json and not
>>>>>>>>> x-ndjson, the scenario of Grafana -> Knox -> ES with multi search
>>>>>>>>> doesn't
>>>>>>>>> work.
>>>>>>>>>
>>>>>>>>> I think we have two solution:
>>>>>>>>>
>>>>>>>>> 1- make some thing to force Grafana change header of POST and put
>>>>>>>>> x-ndjson instead of json, or
>>>>>>>>> 2- rewrite Header by Knox to forwarded with x-ndjson to ES.
>>>>>>>>>
>>>>>>>>> If some one have any idea of how to make this or other solution.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Regards
>>>>>>>>> Rabii
>>>>>>>>>
>>>>>>>>> Le ven. 21 sept. 2018 à 19:21, Kevin Risden <kris...@apache.org>
>>>>>>>>> a écrit :
>>>>>>>>>
>>>>>>>>>> https://github.com/elastic/elasticsearch/issues/25718
>>>>>>>>>>
>>>>>>>>>> Elasticsearch endpoints should take both application/json and
>>>>>>>>>> application/x-ndjson. There might be something else going on here.
>>>>>>>>>> That is
>>>>>>>>>> why I asked for the Elasticsearch logs.
>>>>>>>>>>
>>>>>>>>>> It might not be necessary to rewrite the content type.
>>>>>>>>>>
>>>>>>>>>> Kevin Risden
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Sep 21, 2018 at 1:17 PM rabii lamriq <lam...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi all,
>>>>>>>>>>>
>>>>>>>>>>> My problem is how to change the content-type=application/json to
>>>>>>>>>>> application/x-ndjson by rewriting the header in the knox.
>>>>>>>>>>>
>>>>>>>>>>> Because grafana send their query with content-type=application/json
>>>>>>>>>>> and knox return status 400,
>>>>>>>>>>>
>>>>>>>>>>> Hower when i reproduce the problem by curl comande, I noticed
>>>>>>>>>>> that if i replace json by x-ndjson through knox it wirk well.
>>>>>>>>>>>
>>>>>>>>>>> Sence i can't change the content-type in grafana, is this
>>>>>>>>>>> possible by rewriting the header in knox to replace json by
>>>>>>>>>>> x-ndjson.
>>>>>>>>>>>
>>>>>>>>>>> Regards
>>>>>>>>>>> Rabii
>>>>>>>>>>>
>>>>>>>>>>> Le ven. 21 sept. 2018 à 18:05,
>>>>>>>>>>> Kevin Risden
>>>>>>>>>>> <kris...@apache.org> a écrit :
>>>>>>>>>>>
>>>>>>>>>>>> Do you have the error from Elasticsearch logs? It might explain
>>>>>>>>>>>> if Knox or Grafana is sending a bad request.
>>>>>>>>>>>>
>>>>>>>>>>>> Kevin Risden
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Sep 21, 2018 at 12:03 PM Sandeep Moré <
>>>>>>>>>>>> moresand...@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I don't see any issues from Knox side, what error do you see
>>>>>>>>>>>>> in Grafana ? I think this could be because Grafana is expecting
>>>>>>>>>>>>> certain
>>>>>>>>>>>>> content-type.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Sep 21, 2018 at 11:37 AM rabii lamriq <
>>>>>>>>>>>>> lam...@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Sandeep
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> thanks for your feedback, below is the deboug log for this
>>>>>>>>>>>>>> curl
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2018-09-18 17:23:23,154 DEBUG hadoop.gateway
>>>>>>>>>>>>>> (UrlRewriteProcessor.java:rewrite(166)) - Rewrote URL:
>>>>>>>>>>>>>> https://<host-knox>:8443/gateway/default/es5/<index_name>/_msearch,
>>>>>>>>>>>>>> direction: IN via explicit rule: ES5/inbound/query to URL:
>>>>>>>>>>>>>> http://
>>>>>>>>>>>>>> <host-elast>:9200/<index_name>/_msearch
>>>>>>>>>>>>>> 2018-09-18 17:23:23,156 DEBUG hadoop.gateway
>>>>>>>>>>>>>> (ElasticsearchDispatch.java:executeOutboundRequest(79)) -
>>>>>>>>>>>>>> Dispatch request:
>>>>>>>>>>>>>> POST http://
>>>>>>>>>>>>>> <host-elast>:9200/<index_name>/_msearch?doAs=<u_user>
>>>>>>>>>>>>>> 2018-09-18 17:23:23,193 DEBUG hadoop.gateway
>>>>>>>>>>>>>> (ElasticsearchDispatch.java:executeOutboundRequest(95)) -
>>>>>>>>>>>>>> Dispatch response
>>>>>>>>>>>>>> status: 400
>>>>>>>>>>>>>> 2018-09-18 17:23:23,194 DEBUG hadoop.gateway
>>>>>>>>>>>>>> (DefaultDispatch.java:getInboundResponseContentType(202)) -
>>>>>>>>>>>>>> Using explicit
>>>>>>>>>>>>>> character set UTF-8 for entity of type application/json
>>>>>>>>>>>>>> 2018-09-18 17:23:23,195 DEBUG hadoop.gateway
>>>>>>>>>>>>>> (DefaultDispatch.java:getInboundResponseContentType(210)) -
>>>>>>>>>>>>>> Inbound
>>>>>>>>>>>>>> response entity content type: application/json; charset=UTF-8
>>>>>>>>>>>>>> 2018-09-18 17:23:54,519 DEBUG hadoop.gateway
>>>>>>>>>>>>>> (GatewayFilter.java:doFilter(116)) - Received request: POST
>>>>>>>>>>>>>> /es5/<index_name>/_msearch
>>>>>>>>>>>>>> 2018-09-18 17:23:54,521 INFO hadoop.gateway
>>>>>>>>>>>>>> (AclsAuthorizationFilter.java:doFilter(85)) - Access Granted:
>>>>>>>>>>>>>> true
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>> Rabii
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Le ven. 21 sept. 2018 à 17:17, Sandeep Moré <
>>>>>>>>>>>>>> moresand...@gmail.com> a écrit :
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hello Rab,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Interesting, do you have the debug logs for this ? they will
>>>>>>>>>>>>>>> tell us exactly what is happening.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>> Sandeep
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Sep 21, 2018 at 9:56 AM rabii lamriq <
>>>>>>>>>>>>>>> lam...@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> From Grafana, I Post multisearch query to Elasticsearch
>>>>>>>>>>>>>>>> using the bulk API with the newline delimited JSON (NDJSON)
>>>>>>>>>>>>>>>> format.
>>>>>>>>>>>>>>>> https://www.elastic.co/guide/en/elasticsearch/reference/current/search-multi-search.html
>>>>>>>>>>>>>>>> like this:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> curl -u myuser -i -H “Content-Type: application/json”
>>>>>>>>>>>>>>>> -XPOST "http://host_ES:900/index/_msearch"; --data-binary ’
>>>>>>>>>>>>>>>> { first json doc}
>>>>>>>>>>>>>>>> {second json doc}
>>>>>>>>>>>>>>>> ’
>>>>>>>>>>>>>>>> when bypassing Knox, The query work well either we made
>>>>>>>>>>>>>>>> json or x-ndjson,
>>>>>>>>>>>>>>>> However, through Knox gateway, if the
>>>>>>>>>>>>>>>> content-type=application/json, the query retun this error of
>>>>>>>>>>>>>>>> 400:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> HTTP/1.1 400 Bad Request
>>>>>>>>>>>>>>>> content-type: application/json
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> But when we replace json by x-ndjson like mentioned in the
>>>>>>>>>>>>>>>> bulk API, it work well.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> My problem is that Grafana send their query with the
>>>>>>>>>>>>>>>> content-type=application/json and not x-ndjson, for this, it
>>>>>>>>>>>>>>>> is possible to
>>>>>>>>>>>>>>>> make any thing on Knox to solve this problem?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>>>> Rab
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
