[jira] [Commented] (NIFI-6403) ElasticSearch field selection broken in Elastic 7.0+
[
https://issues.apache.org/jira/browse/NIFI-6403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17239078#comment-17239078
]
ASF subversion and git services commented on NIFI-6403:
---
Commit 124cdbd3fe5446bff08346e5a28f2883f13e2848 in nifi's branch
refs/heads/main from Chris Sampson
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=124cdbd ]
NIFI-6403 and NIFI-6404: Elasticsearch 7 support
Addressed PR#4153 comments; removed ES Version property and made Type optional
in all ES HTTP/Record processors, applying sensible default values where
required; use _source queyr parameter instead of _source_include/s as it's
compatible between ES versions
Fix unit test compilation to use JDK8-compatible library/method
Better optional type and id handling for PutElasticsearchRecord; update
nifi-elasticsearch-client-service build dependencies to use latest versions of
Elasticsearch in each supported major version (5/6/7); addressed several
warnings in ElasticSearchClientServiceImpl
This closes #4667.
Signed-off-by: Koji Kawamura
> ElasticSearch field selection broken in Elastic 7.0+
>
>
> Key: NIFI-6403
> URL: https://issues.apache.org/jira/browse/NIFI-6403
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.10.0, 1.9.2
>Reporter: Wietze B
>Assignee: Chris Sampson
>Priority: Major
> Attachments: NIFI-6403.json, NIFI-6403.xml
>
> Original Estimate: 0.25h
> Time Spent: 10h 10m
> Remaining Estimate: 0h
>
> Elastic has
> [deprecated|https://www.elastic.co/guide/en/elasticsearch/reference/6.6/breaking-changes-6.6.html#_deprecate_literal__source_exclude_literal_and_literal__source_include_literal_url_parameters]
> the {{source_include}} search parameter in favour of {{source_includes}} in
> version 7.0 and higher.
> This means that processors using the field selection will get an HTTP 400
> error upon execution.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-6403) ElasticSearch field selection broken in Elastic 7.0+
[
https://issues.apache.org/jira/browse/NIFI-6403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17239077#comment-17239077
]
ASF subversion and git services commented on NIFI-6403:
---
Commit a3d845a38f93fed1f65ec85c851ca845ef186556 in nifi's branch
refs/heads/main from Joe Gresock
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=a3d845a ]
NIFI-6403: Adding Elasticsearch7 support to HTTP processors
This closes #4153.
Signed-off-by: Koji Kawamura
> ElasticSearch field selection broken in Elastic 7.0+
>
>
> Key: NIFI-6403
> URL: https://issues.apache.org/jira/browse/NIFI-6403
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.10.0, 1.9.2
>Reporter: Wietze B
>Assignee: Chris Sampson
>Priority: Major
> Attachments: NIFI-6403.json, NIFI-6403.xml
>
> Original Estimate: 0.25h
> Time Spent: 10h 10m
> Remaining Estimate: 0h
>
> Elastic has
> [deprecated|https://www.elastic.co/guide/en/elasticsearch/reference/6.6/breaking-changes-6.6.html#_deprecate_literal__source_exclude_literal_and_literal__source_include_literal_url_parameters]
> the {{source_include}} search parameter in favour of {{source_includes}} in
> version 7.0 and higher.
> This means that processors using the field selection will get an HTTP 400
> error upon execution.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-6403) ElasticSearch field selection broken in Elastic 7.0+
[
https://issues.apache.org/jira/browse/NIFI-6403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17236059#comment-17236059
]
Chris Sampson commented on NIFI-6403:
-
Rebased from main and addressed additional comments - updated template/flow
attached to this Jira.
> ElasticSearch field selection broken in Elastic 7.0+
>
>
> Key: NIFI-6403
> URL: https://issues.apache.org/jira/browse/NIFI-6403
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.10.0, 1.9.2
>Reporter: Wietze B
>Assignee: Joseph Gresock
>Priority: Major
> Attachments: NIFI-6403.json, NIFI-6403.xml
>
> Original Estimate: 0.25h
> Time Spent: 8h 10m
> Remaining Estimate: 0h
>
> Elastic has
> [deprecated|https://www.elastic.co/guide/en/elasticsearch/reference/6.6/breaking-changes-6.6.html#_deprecate_literal__source_exclude_literal_and_literal__source_include_literal_url_parameters]
> the {{source_include}} search parameter in favour of {{source_includes}} in
> version 7.0 and higher.
> This means that processors using the field selection will get an HTTP 400
> error upon execution.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-6403) ElasticSearch field selection broken in Elastic 7.0+
[
https://issues.apache.org/jira/browse/NIFI-6403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17232782#comment-17232782
]
Chris Sampson commented on NIFI-6403:
-
Raised [PR 4667|https://github.com/apache/nifi/pull/4667] to address comments
from #4153 and take a slightly different approach.
Attached [template|^NIFI-6403.xml]/[flow|^NIFI-6403.json] for integration
tests with affected processors.
> ElasticSearch field selection broken in Elastic 7.0+
>
>
> Key: NIFI-6403
> URL: https://issues.apache.org/jira/browse/NIFI-6403
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.10.0, 1.9.2
>Reporter: Wietze B
>Assignee: Joseph Gresock
>Priority: Major
> Attachments: NIFI-6403.json, NIFI-6403.xml
>
> Original Estimate: 0.25h
> Time Spent: 7h 40m
> Remaining Estimate: 0h
>
> Elastic has
> [deprecated|https://www.elastic.co/guide/en/elasticsearch/reference/6.6/breaking-changes-6.6.html#_deprecate_literal__source_exclude_literal_and_literal__source_include_literal_url_parameters]
> the {{source_include}} search parameter in favour of {{source_includes}} in
> version 7.0 and higher.
> This means that processors using the field selection will get an HTTP 400
> error upon execution.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-6403) ElasticSearch field selection broken in Elastic 7.0+
[
https://issues.apache.org/jira/browse/NIFI-6403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17108312#comment-17108312
]
Koji Kawamura commented on NIFI-6403:
-
[~jgresock] I found NIFI-6404. Please update the Github PR to state that it
also addresses NIFI-6404. Thanks.
> ElasticSearch field selection broken in Elastic 7.0+
>
>
> Key: NIFI-6403
> URL: https://issues.apache.org/jira/browse/NIFI-6403
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.10.0, 1.9.2
>Reporter: Wietze B
>Assignee: Joseph Gresock
>Priority: Major
> Original Estimate: 0.25h
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Elastic has
> [deprecated|https://www.elastic.co/guide/en/elasticsearch/reference/6.6/breaking-changes-6.6.html#_deprecate_literal__source_exclude_literal_and_literal__source_include_literal_url_parameters]
> the {{source_include}} search parameter in favour of {{source_includes}} in
> version 7.0 and higher.
> This means that processors using the field selection will get an HTTP 400
> error upon execution.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-6403) ElasticSearch field selection broken in Elastic 7.0+
[
https://issues.apache.org/jira/browse/NIFI-6403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17108282#comment-17108282
]
Koji Kawamura commented on NIFI-6403:
-
[~jgresock] Thanks for your contribution! Sorry for it's taking so long for
your PR to get reviewed. I took a brief look at the latest PR today. And found
it covers broader processors to support Elasticsearch 7.x. Would you mind
updating this JIRA description to meet the updates?
> ElasticSearch field selection broken in Elastic 7.0+
>
>
> Key: NIFI-6403
> URL: https://issues.apache.org/jira/browse/NIFI-6403
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.10.0, 1.9.2
>Reporter: Wietze B
>Assignee: Joseph Gresock
>Priority: Major
> Original Estimate: 0.25h
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Elastic has
> [deprecated|https://www.elastic.co/guide/en/elasticsearch/reference/6.6/breaking-changes-6.6.html#_deprecate_literal__source_exclude_literal_and_literal__source_include_literal_url_parameters]
> the {{source_include}} search parameter in favour of {{source_includes}} in
> version 7.0 and higher.
> This means that processors using the field selection will get an HTTP 400
> error upon execution.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-6403) ElasticSearch field selection broken in Elastic 7.0+
[
https://issues.apache.org/jira/browse/NIFI-6403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17062795#comment-17062795
]
Joseph Gresock commented on NIFI-6403:
--
Created a new pull request to resolve merge conflict.
> ElasticSearch field selection broken in Elastic 7.0+
>
>
> Key: NIFI-6403
> URL: https://issues.apache.org/jira/browse/NIFI-6403
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.10.0, 1.9.2
>Reporter: Wietze B
>Assignee: Joseph Gresock
>Priority: Major
> Original Estimate: 0.25h
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Elastic has
> [deprecated|https://www.elastic.co/guide/en/elasticsearch/reference/6.6/breaking-changes-6.6.html#_deprecate_literal__source_exclude_literal_and_literal__source_include_literal_url_parameters]
> the {{source_include}} search parameter in favour of {{source_includes}} in
> version 7.0 and higher.
> This means that processors using the field selection will get an HTTP 400
> error upon execution.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-6403) ElasticSearch field selection broken in Elastic 7.0+
[
https://issues.apache.org/jira/browse/NIFI-6403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17062749#comment-17062749
]
Joseph Gresock commented on NIFI-6403:
--
This also works on nifi 1.11.3 (with a minor merge conflict resolution)
> ElasticSearch field selection broken in Elastic 7.0+
>
>
> Key: NIFI-6403
> URL: https://issues.apache.org/jira/browse/NIFI-6403
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.10.0, 1.9.2
>Reporter: Wietze B
>Assignee: Joseph Gresock
>Priority: Major
> Original Estimate: 0.25h
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Elastic has
> [deprecated|https://www.elastic.co/guide/en/elasticsearch/reference/6.6/breaking-changes-6.6.html#_deprecate_literal__source_exclude_literal_and_literal__source_include_literal_url_parameters]
> the {{source_include}} search parameter in favour of {{source_includes}} in
> version 7.0 and higher.
> This means that processors using the field selection will get an HTTP 400
> error upon execution.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
