[
https://issues.apache.org/jira/browse/CAMEL-26283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen updated CAMEL-26283:
--------------------------------
Description:
The camel-elasticsearch-rest-client producer builds the path of each REST
request with String.format, for example /<index>/_doc/<id>, and places the
index name and the document id into it as they are, without encoding. The
Elasticsearch low-level client then parses that string as a URI. A document id
or index name that contains a reserved character is therefore not sent as the
value it was given:
* a '/' in the id adds a path segment
* a '?' or '#' in the id ends the path, so only the part before it is used as
the id
* a space or a '%' that is not part of a valid escape makes the request fail
while the URI is parsed
The Elasticsearch Java API client sends the same values as single, encoded path
segments, so an id such as "a/b?c#d e%" works there but not with this component.
In addition, GET_BY_ID reads the "found" field of the response as a boolean
primitive, so a response without that field ends in a NullPointerException
instead of returning no document.
h3. Change
In ElasticsearchRestClientProducer:
* add a small helper that encodes a value as one path segment (URLEncoder with
UTF-8, with '+' turned into %20). It rejects an empty value and the values "."
and "..", which cannot be sent as a single segment.
* use it for the document id in indexRequest, getById and delete, and for the
index name in createIndexRequest, deleteIndexRequest, indexRequest, getById,
delete and search.
* a comma separated list of index names and wildcard index patterns (for
example logs-*) keep working, as Elasticsearch decodes the path segment before
it splits the list.
* for the operations on one index (createIndex, deleteIndex, indexRequest,
getById and delete), check the index name against the Elasticsearch index
naming rules before the request is built: lowercase only, none of {{\ / * ? " <
> | , #}} or a space, not starting with {{-}}, {{_}} or {{+}}, and not {{.}} or
{{..}}. A name that breaks them fails with a clear message instead of an error
from the server. search keeps accepting comma separated lists and wildcard
patterns.
* extractDocument returns no document when "found" is missing or false.
The component documentation will say that the document id and the index name
are each sent as one path segment, with reserved characters encoded, so the id
and the index name are always taken literally. The upgrade guides on main
(4.24, and 4.22 / 4.18 for the backports) will note the new encoding, and that
"." and ".." are no longer accepted as a document id.
h3. Tests
* a unit test, ElasticsearchRestClientRequestPathTest, that checks the request
path built for ids with '?', '#', '/', space and '%', for an index name with
reserved characters, that an index name breaking the naming rules is rejected
for the single-index operations, for a comma separated index list, that "." and
".." are rejected, and that a GET_BY_ID response without "found" gives no
document.
* an integration test in ElasticsearchRestClientComponentIT that indexes and
reads back a document whose id contains reserved characters.
The change is limited to encoding and to the "found" check, so it can be
backported to camel-4.22.x and camel-4.18.x. It touches the same methods as
CAMEL-26234 and will be sequenced with it.
was:
The camel-elasticsearch-rest-client producer builds the path of each REST
request with String.format, for example /<index>/_doc/<id>, and places the
index name and the document id into it as they are, without encoding. The
Elasticsearch low-level client then parses that string as a URI. A document id
or index name that contains a reserved character is therefore not sent as the
value it was given:
* a '/' in the id adds a path segment
* a '?' or '#' in the id ends the path, so only the part before it is used as
the id
* a space or a '%' that is not part of a valid escape makes the request fail
while the URI is parsed
The Elasticsearch Java API client sends the same values as single, encoded path
segments, so an id such as "a/b?c#d e%" works there but not with this component.
In addition, GET_BY_ID reads the "found" field of the response as a boolean
primitive, so a response without that field ends in a NullPointerException
instead of returning no document.
h3. Change
In ElasticsearchRestClientProducer:
* add a small helper that encodes a value as one path segment (URLEncoder with
UTF-8, with '+' turned into %20). It rejects an empty value and the values "."
and "..", which cannot be sent as a single segment.
* use it for the document id in indexRequest, getById and delete, and for the
index name in createIndexRequest, deleteIndexRequest, indexRequest, getById,
delete and search.
* a comma separated list of index names and wildcard index patterns (for
example logs-*) keep working, as Elasticsearch decodes the path segment before
it splits the list.
* extractDocument returns no document when "found" is missing or false.
The component documentation will say that the document id and the index name
are each sent as one path segment, with reserved characters encoded, so the id
and the index name are always taken literally. The upgrade guides on main
(4.24, and 4.22 / 4.18 for the backports) will note the new encoding, and that
"." and ".." are no longer accepted as a document id.
h3. Tests
* a unit test, ElasticsearchRestClientRequestPathTest, that checks the request
path built for ids with '?', '#', '/', space and '%', for an index name with
reserved characters, for a comma separated index list, that "." and ".." are
rejected, and that a GET_BY_ID response without "found" gives no document.
* an integration test in ElasticsearchRestClientComponentIT that indexes and
reads back a document whose id contains reserved characters.
The change is limited to encoding and to the "found" check, so it can be
backported to camel-4.22.x and camel-4.18.x. It touches the same methods as
CAMEL-26234 and will be sequenced with it.
> camel-elasticsearch-rest-client - encode document id and index name in the
> path
> -------------------------------------------------------------------------------
>
> Key: CAMEL-26283
> URL: https://issues.apache.org/jira/browse/CAMEL-26283
> Project: Camel
> Issue Type: Improvement
> Components: camel-elasticsearch-rest
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Priority: Major
> Fix For: 4.18.6, 4.22.3, 4.24.0
>
>
> The camel-elasticsearch-rest-client producer builds the path of each REST
> request with String.format, for example /<index>/_doc/<id>, and places the
> index name and the document id into it as they are, without encoding. The
> Elasticsearch low-level client then parses that string as a URI. A document
> id or index name that contains a reserved character is therefore not sent as
> the value it was given:
> * a '/' in the id adds a path segment
> * a '?' or '#' in the id ends the path, so only the part before it is used as
> the id
> * a space or a '%' that is not part of a valid escape makes the request fail
> while the URI is parsed
> The Elasticsearch Java API client sends the same values as single, encoded
> path segments, so an id such as "a/b?c#d e%" works there but not with this
> component.
> In addition, GET_BY_ID reads the "found" field of the response as a boolean
> primitive, so a response without that field ends in a NullPointerException
> instead of returning no document.
> h3. Change
> In ElasticsearchRestClientProducer:
> * add a small helper that encodes a value as one path segment (URLEncoder
> with UTF-8, with '+' turned into %20). It rejects an empty value and the
> values "." and "..", which cannot be sent as a single segment.
> * use it for the document id in indexRequest, getById and delete, and for the
> index name in createIndexRequest, deleteIndexRequest, indexRequest, getById,
> delete and search.
> * a comma separated list of index names and wildcard index patterns (for
> example logs-*) keep working, as Elasticsearch decodes the path segment
> before it splits the list.
> * for the operations on one index (createIndex, deleteIndex, indexRequest,
> getById and delete), check the index name against the Elasticsearch index
> naming rules before the request is built: lowercase only, none of {{\ / * ? "
> < > | , #}} or a space, not starting with {{-}}, {{_}} or {{+}}, and not
> {{.}} or {{..}}. A name that breaks them fails with a clear message instead
> of an error from the server. search keeps accepting comma separated lists and
> wildcard patterns.
> * extractDocument returns no document when "found" is missing or false.
> The component documentation will say that the document id and the index name
> are each sent as one path segment, with reserved characters encoded, so the
> id and the index name are always taken literally. The upgrade guides on main
> (4.24, and 4.22 / 4.18 for the backports) will note the new encoding, and
> that "." and ".." are no longer accepted as a document id.
> h3. Tests
> * a unit test, ElasticsearchRestClientRequestPathTest, that checks the
> request path built for ids with '?', '#', '/', space and '%', for an index
> name with reserved characters, that an index name breaking the naming rules
> is rejected for the single-index operations, for a comma separated index
> list, that "." and ".." are rejected, and that a GET_BY_ID response without
> "found" gives no document.
> * an integration test in ElasticsearchRestClientComponentIT that indexes and
> reads back a document whose id contains reserved characters.
> The change is limited to encoding and to the "found" check, so it can be
> backported to camel-4.22.x and camel-4.18.x. It touches the same methods as
> CAMEL-26234 and will be sequenced with it.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
