[
https://issues.apache.org/jira/browse/NIFI-6404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Koji Kawamura updated NIFI-6404:
--------------------------------
Fix Version/s: 1.13.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> PutElasticsearchHttp: Remove _type as being compulsory
> ------------------------------------------------------
>
> Key: NIFI-6404
> URL: https://issues.apache.org/jira/browse/NIFI-6404
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Affects Versions: 1.10.0, 1.9.2
> Environment: Elasticsearch 7.x
> Reporter: David Vassallo
> Assignee: Joseph Gresock
> Priority: Major
> Fix For: 1.13.0
>
>
> In ES 7.x and above, document "type" is no longer compulsory and in fact is
> deprecated. When using the 1.9.2 version of PutElasticsearchHttp with ES
> v7.2, it still works however you'll see the following HTTP in the response:
>
> {{HTTP/1.1 200 OK}}
> *{{Warning: 299 Elasticsearch-7.2.0-508c38a "[types removal] Specifying
> types in bulk requests is deprecated."}}*
> {{content-type: application/json; charset=UTF-8}}
>
> The fix is relatively straightforward:
> * In *PutElasticserachHttp.java*, remove the requirement of a compulsory
> "Type" property:
> {code:java}
> public static final PropertyDescriptor TYPE = new PropertyDescriptor.Builder()
> .name("put-es-type")
> .displayName("Type")
> .description("The type of this document (used by Elasticsearch < 7.0 for
> indexing and searching). Leave empty for ES >= 7.0") // <-
> .required(false) // <--------- CHANGE
> .expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
> .addValidator(StandardValidators.NON_EMPTY_EL_VALIDATOR)
> .build();
> {code}
>
> * In *AbstractElasticsearchHttpProcessor.java*, check for the presence of
> "docType". If not present, assume elasticsearch 7.x or above and omit from
> bulk API URL:
>
> {code:java}
> protected void buildBulkCommand(StringBuilder sb, String index, String
> docType, String indexOp, String id, String jsonString) {
> if (indexOp.equalsIgnoreCase("index")) {
> sb.append("{\"index\": { \"_index\": \"");
> sb.append(StringEscapeUtils.escapeJson(index));
> if (!(StringUtils.isEmpty(docType) | docType == null)){ // <---------
> CHANGE START
> sb.append("\", \"_type\": \"");
> sb.append(StringEscapeUtils.escapeJson(docType));
> sb.append("\"");
> } // <--------- CHANGE END
> if (!StringUtils.isEmpty(id)) {
> sb.append(", \"_id\": \"");
> sb.append(StringEscapeUtils.escapeJson(id));
> sb.append("\"");
> }
> sb.append("}}\n");
> sb.append(jsonString);
> sb.append("\n");
> } else if (indexOp.equalsIgnoreCase("upsert") ||
> indexOp.equalsIgnoreCase("update")) {
> sb.append("{\"update\": { \"_index\": \"");
> sb.append(StringEscapeUtils.escapeJson(index));
> sb.append("\", \"_type\": \"");
> sb.append(StringEscapeUtils.escapeJson(docType));
> sb.append("\", \"_id\": \"");
> sb.append(StringEscapeUtils.escapeJson(id));
> sb.append("\" }\n");
> sb.append("{\"doc\": ");
> sb.append(jsonString);
> sb.append(", \"doc_as_upsert\": ");
> sb.append(indexOp.equalsIgnoreCase("upsert"));
> sb.append(" }\n");
> } else if (indexOp.equalsIgnoreCase("delete")) {
> sb.append("{\"delete\": { \"_index\": \"");
> sb.append(StringEscapeUtils.escapeJson(index));
> sb.append("\", \"_type\": \"");
> sb.append(StringEscapeUtils.escapeJson(docType));
> sb.append("\", \"_id\": \"");
> sb.append(StringEscapeUtils.escapeJson(id));
> sb.append("\" }\n");
> }
> }
> {code}
>
> * The *TestPutElasticsearchHttp.java* test file needs to be updated to
> reflect that now a requests without type is valid (it's currently marked as
> invalid)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
