[jira] [Created] (NIFI-6403) ElasticSearch field selection broken in Elastic 7.0+
Wietze B created NIFI-6403:
--
Summary: ElasticSearch field selection broken in Elastic 7.0+
Key: NIFI-6403
URL: https://issues.apache.org/jira/browse/NIFI-6403
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 1.9.2
Reporter: Wietze B
Elastic has
[deprecated|https://www.elastic.co/guide/en/elasticsearch/reference/6.6/breaking-changes-6.6.html#_deprecate_literal__source_exclude_literal_and_literal__source_include_literal_url_parameters]
the {{source_include}} search parameter in favour of {{source_includes}} in
version 7.0 and higher.
This means that processors using the field selection will get an HTTP 400 error
upon execution.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi issue #2942: NIFI-5500: Array support in QueryElasticseachHttp
Github user wietze commented on the issue: https://github.com/apache/nifi/pull/2942 @jzonthemtn Fixed/optimised this in bc3712d ---
[GitHub] nifi pull request #2942: NIFI-5500
GitHub user wietze opened a pull request: https://github.com/apache/nifi/pull/2942 NIFI-5500 Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [x] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [x] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/wietze/nifi NIFI-5500 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2942.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2942 commit 25e2ee280e0b856b73de8cd0c8f039521a883f32 Author: Wietze Date: 2018-08-08T16:17:47Z Adding support for array fields when outputting to attributes when using QueryElasticsearchHttp ---
[jira] [Created] (NIFI-5500) Add array support to QueryElasticsearchHttp
Wietze B created NIFI-5500: -- Summary: Add array support to QueryElasticsearchHttp Key: NIFI-5500 URL: https://issues.apache.org/jira/browse/NIFI-5500 Project: Apache NiFi Issue Type: Improvement Affects Versions: 1.7.1 Reporter: Wietze B When using the QueryElasticsearchHttp processor (with output=Attributes) to query a document that contains a field with an Array rather than a String or Integer, the resulting attribute for that field will be an empty string. This is due to the fact that the QueryElasticsearchHttp component doesn't handle array fields correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-5434) ExecuteScript/Jython imports fail randomly
[
https://issues.apache.org/jira/browse/NIFI-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wietze B updated NIFI-5434:
---
Description:
I have a number of ExecuteScript processors running Jython that import
libraries such as {{threading}}, {{urllib3}}, {{requests}}, etc.
When I boot up my machine or initially start the NiFi service, the
ExecuteScript processors relying on external libraries usually to don't work
straight away, throwing {{AttributeError}} s or {{ImportError}} s on libraries
that definitely exist and are accessible. Restarting the machine or the NiFi
service again usually fixes the problem, but it is unclear to me why the
imports sometimes fail, and sometimes work.
See attached two screenshots and an extract from the logs.
was:
I have a number of ExecuteScript processors running Jython that import
libraries such as {{threading}}, {{urllib3}}, {{requests}}, etc.
When I boot up my machine or initially start the NiFi service, the
ExecuteScript processors relying on external libraries usually to don't work
straight away, throwing {{AttributeError}} s or {{ImportError}} s on libraries
that definitely exist and are accessible. Restarting the machine or the NiFi
service again usually fixes the problem, but it is unclear to me why the
imports sometime fail, and sometimes work.
See attached two screenshots and an extract from the logs.
> ExecuteScript/Jython imports fail randomly
> --
>
> Key: NIFI-5434
> URL: https://issues.apache.org/jira/browse/NIFI-5434
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.6.0, 1.7.0, 1.7.1
> Environment: Debian Stretch (9.5)
>Reporter: Wietze B
>Priority: Major
> Attachments: ibm.png, java_thread.png, log_extract.log
>
>
> I have a number of ExecuteScript processors running Jython that import
> libraries such as {{threading}}, {{urllib3}}, {{requests}}, etc.
> When I boot up my machine or initially start the NiFi service, the
> ExecuteScript processors relying on external libraries usually to don't work
> straight away, throwing {{AttributeError}} s or {{ImportError}} s on
> libraries that definitely exist and are accessible. Restarting the machine or
> the NiFi service again usually fixes the problem, but it is unclear to me why
> the imports sometimes fail, and sometimes work.
> See attached two screenshots and an extract from the logs.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-5434) ExecuteScript/Jython imports fail randomly
[
https://issues.apache.org/jira/browse/NIFI-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wietze B updated NIFI-5434:
---
Description:
I have a number of ExecuteScript processors running Jython that import
libraries such as {{threading}}, {{urllib3}}, {{requests}}, etc.
When I boot up my machine or initially start the NiFi service, the
ExecuteScript processors relying on external libraries usually to don't work
straight away, throwing {{AttributeError}} s or {{ImportError}} s on libraries
that definitely exist and are accessible. Restarting the machine or the NiFi
service again usually fixes the problem, but it is unclear to me why the
imports sometime fail, and sometimes work.
See attached two screenshots and an extract from the logs.
was:
I have a number of ExecuteScript processors running Jython that import
libraries such as {{threading}}, {{urllib3}}, {{requests}}, etc.
When I boot up my machine or initially start the NiFi service, the
ExecuteScript processors relying on external libraries usually to don't work
straight away, throwing {{AttributeError}}s or {{ImportError}}s on libraries
that definitely exist and are accessible. Restarting the machine or the NiFi
service again usually fixes the problem, but it is unclear to me why the
imports sometime fail, and sometimes work.
See attached two screenshots and an extract from the logs.
> ExecuteScript/Jython imports fail randomly
> --
>
> Key: NIFI-5434
> URL: https://issues.apache.org/jira/browse/NIFI-5434
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.6.0, 1.7.0, 1.7.1
> Environment: Debian Stretch (9.5)
>Reporter: Wietze B
>Priority: Major
> Attachments: ibm.png, java_thread.png, log_extract.log
>
>
> I have a number of ExecuteScript processors running Jython that import
> libraries such as {{threading}}, {{urllib3}}, {{requests}}, etc.
> When I boot up my machine or initially start the NiFi service, the
> ExecuteScript processors relying on external libraries usually to don't work
> straight away, throwing {{AttributeError}} s or {{ImportError}} s on
> libraries that definitely exist and are accessible. Restarting the machine or
> the NiFi service again usually fixes the problem, but it is unclear to me why
> the imports sometime fail, and sometimes work.
> See attached two screenshots and an extract from the logs.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (NIFI-5434) ExecuteScript/Jython imports fail randomly
Wietze B created NIFI-5434:
--
Summary: ExecuteScript/Jython imports fail randomly
Key: NIFI-5434
URL: https://issues.apache.org/jira/browse/NIFI-5434
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Affects Versions: 1.7.0, 1.6.0, 1.7.1
Environment: Debian Stretch (9.5)
Reporter: Wietze B
Attachments: ibm.png, java_thread.png, log_extract.log
I have a number of ExecuteScript processors running Jython that import
libraries such as {{threading}}, {{urllib3}}, {{requests}}, etc.
When I boot up my machine or initially start the NiFi service, the
ExecuteScript processors relying on external libraries usually to don't work
straight away, throwing {{AttributeError}}s or {{ImportError}}s on libraries
that definitely exist and are accessible. Restarting the machine or the NiFi
service again usually fixes the problem, but it is unclear to me why the
imports sometime fail, and sometimes work.
See attached two screenshots and an extract from the logs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-3576) QueryElasticsearchHttp should have a "Not Found"/"Zero results" relationship
[ https://issues.apache.org/jira/browse/NIFI-3576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418776#comment-16418776 ] Wietze B commented on NIFI-3576: Are there plans to fix this issue anytime soon? I can confirm the problem is still there. I can't think of an easy workaround either, as the flow literally stops when Elastic doesn't return results. > QueryElasticsearchHttp should have a "Not Found"/"Zero results" relationship > > > Key: NIFI-3576 > URL: https://issues.apache.org/jira/browse/NIFI-3576 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joseph Percivall >Priority: Minor > > In the event of a successful call, QueryElasticsearchHttp always drops the > incoming flowfile and then emits pages of results to the success > relationship. If the search returns no results then no pages of results are > emitted to the success relationship. > The processor should offer other options for handling when there are no > results returned. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
