[jira] [Commented] (NIFI-4927) Create InfluxDB Query Processor
[ https://issues.apache.org/jira/browse/NIFI-4927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427868#comment-16427868 ] ASF GitHub Bot commented on NIFI-4927: -- Github user mans2singh commented on the issue: https://github.com/apache/nifi/pull/2562 Thanks @MikeThomsen for your reveiw/advice. Also thanks @timhallinflux for the pointers on influxdb sandbox. Please let me know if there is other feedback. Mans > Create InfluxDB Query Processor > --- > > Key: NIFI-4927 > URL: https://issues.apache.org/jira/browse/NIFI-4927 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.5.0 >Reporter: Mans Singh >Assignee: Mans Singh >Priority: Minor > Labels: measurements,, query, realtime, timeseries > > Create InfluxDB Query processor -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2562: NIFI-4927 - InfluxDB Query Processor
Github user mans2singh commented on the issue: https://github.com/apache/nifi/pull/2562 Thanks @MikeThomsen for your reveiw/advice. Also thanks @timhallinflux for the pointers on influxdb sandbox. Please let me know if there is other feedback. Mans ---
[GitHub] nifi-minifi-cpp issue #295: MINFICPP-403: Flow Meta tagging
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/295 @minifirocks that script is used with cmake to provide that information in a codified, statically defined way, to the code base. It's built at compile time and provides the agent with the information. Instead of using pragma definitions to pass the information it is statically defined. Additionally, the information this ticket is supposed to provide is flow information not agent information, correct? ---
[GitHub] nifi-minifi-cpp issue #295: MINFICPP-403: Flow Meta tagging
Github user minifirocks commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/295 @phrocker Thanks for the review. the PR is not only for version, it provide a flexible framework to add other meta info also. also in the cmake file, we already specify major/minor/patch version, why we need another generateVersion.sh. ---
[jira] [Commented] (NIFI-4927) Create InfluxDB Query Processor
[ https://issues.apache.org/jira/browse/NIFI-4927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427796#comment-16427796 ] ASF GitHub Bot commented on NIFI-4927: -- Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2562 Forgot to add that a link to a comprehensive sandbox is appreciated @timhallinflux > Create InfluxDB Query Processor > --- > > Key: NIFI-4927 > URL: https://issues.apache.org/jira/browse/NIFI-4927 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.5.0 >Reporter: Mans Singh >Assignee: Mans Singh >Priority: Minor > Labels: measurements,, query, realtime, timeseries > > Create InfluxDB Query processor -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2562: NIFI-4927 - InfluxDB Query Processor
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2562 Forgot to add that a link to a comprehensive sandbox is appreciated @timhallinflux ---
[jira] [Commented] (NIFI-4927) Create InfluxDB Query Processor
[ https://issues.apache.org/jira/browse/NIFI-4927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427795#comment-16427795 ] ASF GitHub Bot commented on NIFI-4927: -- Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2562 @timhallinflux @mans2singh already provided a decent enough Docker Compose test for it. So unless @joewitt or someone else sees something that I missed that's a show stopper, I think we're good to go on merging this to master when 1.6 is released into the wild. > Create InfluxDB Query Processor > --- > > Key: NIFI-4927 > URL: https://issues.apache.org/jira/browse/NIFI-4927 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.5.0 >Reporter: Mans Singh >Assignee: Mans Singh >Priority: Minor > Labels: measurements,, query, realtime, timeseries > > Create InfluxDB Query processor -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2562: NIFI-4927 - InfluxDB Query Processor
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2562 @timhallinflux @mans2singh already provided a decent enough Docker Compose test for it. So unless @joewitt or someone else sees something that I missed that's a show stopper, I think we're good to go on merging this to master when 1.6 is released into the wild. ---
[jira] [Commented] (NIFI-4927) Create InfluxDB Query Processor
[ https://issues.apache.org/jira/browse/NIFI-4927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427787#comment-16427787 ] ASF GitHub Bot commented on NIFI-4927: -- Github user timhallinflux commented on the issue: https://github.com/apache/nifi/pull/2562 @joewitt and @MikeThomsen -- you can also just quickly spin up the TICK stack via the sandbox. Have a look here... https://github.com/influxdata/sandbox If @joewitt is no longer technical enough to run this, I'll be more than happy to do a screen share and help him out! :-) Thank you @mans2singh for contributing this! > Create InfluxDB Query Processor > --- > > Key: NIFI-4927 > URL: https://issues.apache.org/jira/browse/NIFI-4927 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.5.0 >Reporter: Mans Singh >Assignee: Mans Singh >Priority: Minor > Labels: measurements,, query, realtime, timeseries > > Create InfluxDB Query processor -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2562: NIFI-4927 - InfluxDB Query Processor
Github user timhallinflux commented on the issue: https://github.com/apache/nifi/pull/2562 @joewitt and @MikeThomsen -- you can also just quickly spin up the TICK stack via the sandbox. Have a look here... https://github.com/influxdata/sandbox If @joewitt is no longer technical enough to run this, I'll be more than happy to do a screen share and help him out! :-) Thank you @mans2singh for contributing this! ---
[jira] [Commented] (NIFI-5020) Consider treating test resources/utilities as a main module
[ https://issues.apache.org/jira/browse/NIFI-5020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427694#comment-16427694 ] Otto Fowler commented on NIFI-5020: --- Would the proposed testing module be at the top level of the project? What would you propose as the name? Would you want to have tasks on this Jira that identify the items to move, maybe per module ( such as standard-processors ) ? or would you want follow on issues that move things as identified? > Consider treating test resources/utilities as a main module > --- > > Key: NIFI-5020 > URL: https://issues.apache.org/jira/browse/NIFI-5020 > Project: Apache NiFi > Issue Type: Task > Components: Tools and Build >Reporter: Aldrin Piri >Priority: Major > > NIFI-5013 took care of consolidating duplicate resources by making a test-jar > available. However, as reported on the mailing list, when building with > maven.test.skip (this is distinct from skipTests), the test artifact will not > be built and preclude projects that depend upon it (even in the scenario when > all tests are skipped) from building correctly. > Source mailing list thread: > https://lists.apache.org/thread.html/ff4e44d1c40519959fbff579bad587a458833c703b5162248bf8812b@%3Cdev.nifi.apache.org%3E -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5020) Consider treating test resources/utilities as a main module
[ https://issues.apache.org/jira/browse/NIFI-5020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427692#comment-16427692 ] Otto Fowler commented on NIFI-5020: --- [https://maven.apache.org/plugins-archives/maven-surefire-plugin-2.12.4/examples/skipping-test.html] Seems to imply that skipTests is preferred. It is what I personally do, and what we do on the METRON project. Is there a compelling reason to support maven.test.skip? If so can you explain a bit? > Consider treating test resources/utilities as a main module > --- > > Key: NIFI-5020 > URL: https://issues.apache.org/jira/browse/NIFI-5020 > Project: Apache NiFi > Issue Type: Task > Components: Tools and Build >Reporter: Aldrin Piri >Priority: Major > > NIFI-5013 took care of consolidating duplicate resources by making a test-jar > available. However, as reported on the mailing list, when building with > maven.test.skip (this is distinct from skipTests), the test artifact will not > be built and preclude projects that depend upon it (even in the scenario when > all tests are skipped) from building correctly. > Source mailing list thread: > https://lists.apache.org/thread.html/ff4e44d1c40519959fbff579bad587a458833c703b5162248bf8812b@%3Cdev.nifi.apache.org%3E -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-5022) Create an AWS Gateway Web API version of InvokeHTTP
[ https://issues.apache.org/jira/browse/NIFI-5022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Otto Fowler updated NIFI-5022: -- Labels: (was: patch-available) Status: Patch Available (was: Open) > Create an AWS Gateway Web API version of InvokeHTTP > --- > > Key: NIFI-5022 > URL: https://issues.apache.org/jira/browse/NIFI-5022 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Otto Fowler >Assignee: Otto Fowler >Priority: Major > > Currently the AWS processors are lacking support to call AWS Gateway Web Apis. > Nifi should provide support for calling this apis, including support for all > the same authentication methods available to the other AWS client processors. > Since these APIs are web services, their expected use would require an > interface more like InvokeHTTP however, than the specialized interfaces for > the other AWS Services. > What would be required then would be a new AWS Processor that exposed the > same interface as InvokeHTTP, but backed by the AWS client support ( and of > course modified to fit the differences between the OK http client and the > Amazon client ). > This new processor should be able to pass all the applicable tests available > in the InvokeHttp test suite. > The processor should also be factored in such a way as to make it possible > for the creation of custom processors for specific apis > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5044) SelectHiveQL accept only one statement
[
https://issues.apache.org/jira/browse/NIFI-5044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427647#comment-16427647
]
Matt Burgess commented on NIFI-5044:
That seems pretty helpful (I had forgotten about msck), the properties could
accept Expression Language so you could include them as attributes on the flow
file which can also contain the query. We can assume that the pre- and
post-statements are not queries (i.e. do not return ResultSets). Are there
HiveQL statements that can contain their own "parameters" which might look like
Expression Language? For example is it possible to say "SELECT * from myTable
where myColumn = ${some.hive.property}" where some.hive.property would be
evaluated on the HiveServer2 side? If not then EL support would be pretty
powerful on the NiFi side. [~pvillard] what do you think?
> SelectHiveQL accept only one statement
> --
>
> Key: NIFI-5044
> URL: https://issues.apache.org/jira/browse/NIFI-5044
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.2.0
>Reporter: Davide Isoardi
>Priority: Critical
>
> In [this
> |[https://github.com/apache/nifi/commit/bbc714e73ba245de7bc32fd9958667c847101f7d]
> ] commit claims to add support to running multiple statements both on
> SelectHiveQL and PutHiveQL; instead, it adds only the support to PutHiveQL,
> so SelectHiveQL still lacks this important feature. @Matt Burgess, I saw that
> you worked on that, is there any reason for this? If not, can we support it?
> If I try to execute this query:
> {quote}set hive.vectorized.execution.enabled = false; SELECT * FROM table_name
> {quote}
> I have this error:
>
> {quote}2018-04-05 13:35:40,572 ERROR [Timer-Driven Process Thread-146]
> o.a.nifi.processors.hive.SelectHiveQL
> SelectHiveQL[id=243d4c17-b1fe-14af--ee8ce15e] Unable to execute
> HiveQL select query set hive.vectorized.execution.enabled = false; SELECT *
> FROM table_name for
> StandardFlowFileRecord[uuid=0e035558-07ce-473b-b0d4-ac00b8b1df93,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1522824912161-2753,
> container=default, section=705], offset=838441,
> length=25],offset=0,name=cliente_attributi.csv,size=25] due to
> org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
> The query did not generate a result set!; routing to failure: {}
> org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
> The query did not generate a result set!
> at
> org.apache.nifi.processors.hive.SelectHiveQL$2.process(SelectHiveQL.java:305)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2529)
> at
> org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:275)
> at
> org.apache.nifi.processors.hive.SelectHiveQL.lambda$onTrigger$0(SelectHiveQL.java:215)
> at
> org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:114)
> at
> org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:106)
> at
> org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:215)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.sql.SQLException: The query did not generate a result set!
> at org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:438)
> at
> org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
> at
> org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
> at
> org.apache.nifi.processors.hive.SelectHiveQL$2.process(SelectHiveQL.java:293)
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5044) SelectHiveQL accept only one statement
[
https://issues.apache.org/jira/browse/NIFI-5044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427630#comment-16427630
]
Davide Isoardi commented on NIFI-5044:
--
Thanks @Matt and @Pierre for looking at this.
In my experience developing NiFi flows, I would need to run arbitrary
statements before and after an Hive query. The {{set}} statement is just an
example of what might be useful. Other statements we often uses are:
- set X=Y
- msck repair table table_name
- ADD JAR hdfs:///tmp/my_jar.jar;
So I would suggest to add 1 property containing the statements to be run before
the query and maybe also 1 property containing others to be run after the
query. What do you think?
> SelectHiveQL accept only one statement
> --
>
> Key: NIFI-5044
> URL: https://issues.apache.org/jira/browse/NIFI-5044
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.2.0
>Reporter: Davide Isoardi
>Priority: Critical
>
> In [this
> |[https://github.com/apache/nifi/commit/bbc714e73ba245de7bc32fd9958667c847101f7d]
> ] commit claims to add support to running multiple statements both on
> SelectHiveQL and PutHiveQL; instead, it adds only the support to PutHiveQL,
> so SelectHiveQL still lacks this important feature. @Matt Burgess, I saw that
> you worked on that, is there any reason for this? If not, can we support it?
> If I try to execute this query:
> {quote}set hive.vectorized.execution.enabled = false; SELECT * FROM table_name
> {quote}
> I have this error:
>
> {quote}2018-04-05 13:35:40,572 ERROR [Timer-Driven Process Thread-146]
> o.a.nifi.processors.hive.SelectHiveQL
> SelectHiveQL[id=243d4c17-b1fe-14af--ee8ce15e] Unable to execute
> HiveQL select query set hive.vectorized.execution.enabled = false; SELECT *
> FROM table_name for
> StandardFlowFileRecord[uuid=0e035558-07ce-473b-b0d4-ac00b8b1df93,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1522824912161-2753,
> container=default, section=705], offset=838441,
> length=25],offset=0,name=cliente_attributi.csv,size=25] due to
> org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
> The query did not generate a result set!; routing to failure: {}
> org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
> The query did not generate a result set!
> at
> org.apache.nifi.processors.hive.SelectHiveQL$2.process(SelectHiveQL.java:305)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2529)
> at
> org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:275)
> at
> org.apache.nifi.processors.hive.SelectHiveQL.lambda$onTrigger$0(SelectHiveQL.java:215)
> at
> org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:114)
> at
> org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:106)
> at
> org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:215)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.sql.SQLException: The query did not generate a result set!
> at org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:438)
> at
> org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
> at
> org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
> at
> org.apache.nifi.processors.hive.SelectHiveQL$2.process(SelectHiveQL.java:293)
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFIREG-158) Should be able to retrieve a flow by id without the bucket it
[
https://issues.apache.org/jira/browse/NIFIREG-158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427570#comment-16427570
]
ASF GitHub Bot commented on NIFIREG-158:
GitHub user bbende opened a pull request:
https://github.com/apache/nifi-registry/pull/108
NIFIREG-158 Added ability to retrieve flow directly by id without kno…
…wing the bucket
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/bbende/nifi-registry NIFIREG-158
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi-registry/pull/108.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #108
commit 216f325dc6d15134207e32db3a3c5af6c17d504e
Author: Bryan Bende
Date: 2018-04-05T20:22:08Z
NIFIREG-158 Added ability to retrieve flow directly by id without knowing
the bucket
> Should be able to retrieve a flow by id without the bucket it
> -
>
> Key: NIFIREG-158
> URL: https://issues.apache.org/jira/browse/NIFIREG-158
> Project: NiFi Registry
> Issue Type: Improvement
>Affects Versions: 0.1.0
>Reporter: Bryan Bende
>Assignee: Bryan Bende
>Priority: Minor
>
> Currently all the flow end-points are nested under a bucket, like:
> {code:java}
> /buckets/{bucketId}/flows/{flowId}
> /buckets/{bucketId}/flows/{flowId}/versions/{version}{code}
> The flow identifier is unique across all buckets, so once the flow is create
> we should be able to support end-points without the bucket id:
> {code:java}
> /flows/{flowId}
> /flows/{flowId}/versions/{version}
> {code}
> We still need to look at the bucket and authorize the bucket id when secure,
> but we'll have the bucket id from retrieving the flow.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi-registry pull request #108: NIFIREG-158 Added ability to retrieve flow ...
GitHub user bbende opened a pull request: https://github.com/apache/nifi-registry/pull/108 NIFIREG-158 Added ability to retrieve flow directly by id without kno⦠â¦wing the bucket You can merge this pull request into a Git repository by running: $ git pull https://github.com/bbende/nifi-registry NIFIREG-158 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-registry/pull/108.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #108 commit 216f325dc6d15134207e32db3a3c5af6c17d504e Author: Bryan Bende Date: 2018-04-05T20:22:08Z NIFIREG-158 Added ability to retrieve flow directly by id without knowing the bucket ---
[jira] [Updated] (NIFI-5045) PutHiveQL routes parse errors to retry instead of failure
[ https://issues.apache.org/jira/browse/NIFI-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-5045: -- Resolution: Fixed Fix Version/s: 1.7.0 Status: Resolved (was: Patch Available) > PutHiveQL routes parse errors to retry instead of failure > - > > Key: NIFI-5045 > URL: https://issues.apache.org/jira/browse/NIFI-5045 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Minor > Fix For: 1.7.0 > > > When the input to PutHiveQL is an invalid HiveQL statement, the documentation > for the failure relationship should apply: "A FlowFile is routed to this > relationship if the database cannot be updated and retrying the operation > will also fail, such as an invalid query or an integrity constraint > violation". However, upon invalid HiveQL, the processor routes the flowfile > to 'retry', which is neither desired or logical (the query is bad, how could > it ever succeed on retry?) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5045) PutHiveQL routes parse errors to retry instead of failure
[ https://issues.apache.org/jira/browse/NIFI-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427518#comment-16427518 ] ASF subversion and git services commented on NIFI-5045: --- Commit 7ff38f690d54cdc607383f1a4be252395b8c5c27 in nifi's branch refs/heads/master from [~ca9mbu] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=7ff38f6 ] NIFI-5045: Fixed error code handling in PutHiveQL. This closes #2608 > PutHiveQL routes parse errors to retry instead of failure > - > > Key: NIFI-5045 > URL: https://issues.apache.org/jira/browse/NIFI-5045 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Minor > Fix For: 1.7.0 > > > When the input to PutHiveQL is an invalid HiveQL statement, the documentation > for the failure relationship should apply: "A FlowFile is routed to this > relationship if the database cannot be updated and retrying the operation > will also fail, such as an invalid query or an integrity constraint > violation". However, upon invalid HiveQL, the processor routes the flowfile > to 'retry', which is neither desired or logical (the query is bad, how could > it ever succeed on retry?) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5045) PutHiveQL routes parse errors to retry instead of failure
[ https://issues.apache.org/jira/browse/NIFI-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427520#comment-16427520 ] ASF GitHub Bot commented on NIFI-5045: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2608 > PutHiveQL routes parse errors to retry instead of failure > - > > Key: NIFI-5045 > URL: https://issues.apache.org/jira/browse/NIFI-5045 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Minor > Fix For: 1.7.0 > > > When the input to PutHiveQL is an invalid HiveQL statement, the documentation > for the failure relationship should apply: "A FlowFile is routed to this > relationship if the database cannot be updated and retrying the operation > will also fail, such as an invalid query or an integrity constraint > violation". However, upon invalid HiveQL, the processor routes the flowfile > to 'retry', which is neither desired or logical (the query is bad, how could > it ever succeed on retry?) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2608: NIFI-5045: Fixed error code handling in PutHiveQL
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2608 ---
[jira] [Commented] (NIFI-5045) PutHiveQL routes parse errors to retry instead of failure
[ https://issues.apache.org/jira/browse/NIFI-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427519#comment-16427519 ] ASF GitHub Bot commented on NIFI-5045: -- Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2608 Thanks @mattyb149. This has been merged to master. > PutHiveQL routes parse errors to retry instead of failure > - > > Key: NIFI-5045 > URL: https://issues.apache.org/jira/browse/NIFI-5045 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Minor > Fix For: 1.7.0 > > > When the input to PutHiveQL is an invalid HiveQL statement, the documentation > for the failure relationship should apply: "A FlowFile is routed to this > relationship if the database cannot be updated and retrying the operation > will also fail, such as an invalid query or an integrity constraint > violation". However, upon invalid HiveQL, the processor routes the flowfile > to 'retry', which is neither desired or logical (the query is bad, how could > it ever succeed on retry?) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2608: NIFI-5045: Fixed error code handling in PutHiveQL
Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2608 Thanks @mattyb149. This has been merged to master. ---
[jira] [Commented] (MINIFICPP-445) Implement escape/unescape CSV functions in expression language
[
https://issues.apache.org/jira/browse/MINIFICPP-445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427509#comment-16427509
]
ASF GitHub Bot commented on MINIFICPP-445:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/293#discussion_r179582611
--- Diff: libminifi/include/utils/StringUtils.h ---
@@ -88,7 +91,10 @@ class StringUtils {
*/
static inline bool equalsIgnoreCase(const std::string &left, const
std::string right) {
if (left.length() == right.length()) {
- return std::equal(right.begin(), right.end(), left.begin(),
[](unsigned char lc, unsigned char rc) {return std::tolower(lc) ==
std::tolower(rc);});
--- End diff --
I noticed you changed a lot of files outside of core. Can you keep these at
200 characters so we avoid unnecessary change later?
> Implement escape/unescape CSV functions in expression language
> --
>
> Key: MINIFICPP-445
> URL: https://issues.apache.org/jira/browse/MINIFICPP-445
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Andrew Christianson
>Assignee: Andrew Christianson
>Priority: Major
>
> *
> [escapeCsv|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#escapecsv]
> *
> [unescapeCsv|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#unescapecsv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #293: MINIFICPP-445 Added escape/unescape CSV e...
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/293#discussion_r179582611
--- Diff: libminifi/include/utils/StringUtils.h ---
@@ -88,7 +91,10 @@ class StringUtils {
*/
static inline bool equalsIgnoreCase(const std::string &left, const
std::string right) {
if (left.length() == right.length()) {
- return std::equal(right.begin(), right.end(), left.begin(),
[](unsigned char lc, unsigned char rc) {return std::tolower(lc) ==
std::tolower(rc);});
--- End diff --
I noticed you changed a lot of files outside of core. Can you keep these at
200 characters so we avoid unnecessary change later?
---
[jira] [Assigned] (NIFI-4997) Actions taken on process groups do not appear in flow configuration history
[ https://issues.apache.org/jira/browse/NIFI-4997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman reassigned NIFI-4997: - Assignee: Matt Gilman > Actions taken on process groups do not appear in flow configuration history > --- > > Key: NIFI-4997 > URL: https://issues.apache.org/jira/browse/NIFI-4997 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.4.0 > Environment: CentOS 7, Docker >Reporter: Craig Becker >Assignee: Matt Gilman >Priority: Major > > Selecting a process group and executing a stop or start action does not cause > anything to be logged in the flow configuration history. The current behavior > makes it impossible to trace who/when a process group was turned off. > > The expected/desired behavior would be to either add a log entry per > processor that was stopped/started, or to log that the process group was > stopped/started. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp issue #295: MINFICPP-403: Flow Meta tagging
Github user phrocker commented on the issue:
https://github.com/apache/nifi-minifi-cpp/pull/295
`#ifndef AGENT_BUILD_H
#define AGENT_BUILD_H
#include
namespace org {
namespace apache {
namespace nifi {
namespace minifi {
class AgentBuild {
public:
static constexpr const char* VERSION = "0.5.0";
static constexpr const char* BUILD_IDENTIFIER = "";
static constexpr const char* BUILD_REV =
"b68d22ba55cfea069cbdbf7931e2b234d86b1f12";
static constexpr const char* BUILD_DATE = "1522951804";
static constexpr const char* COMPILER =
"/Library/Developer/CommandLineTools/usr/bin/c++";
static constexpr const char* COMPILER_VERSION = "9.0.0.938";
static constexpr const char* COMPILER_FLAGS = " -std=c++11 -Wall";
static std::vector getExtensions() {
static std::vector extensions;
if (extensions.empty()){
extensions.push_back("minifi-expression-language-extensions");
extensions.push_back("minifi-http-curl");
extensions.push_back("minifi-civet-extensions");
extensions.push_back("minifi-rocksdb-repos");
extensions.push_back("minifi-archive-extensions");
extensions.push_back("minifi-gps");
extensions.push_back("minifi-mqtt-extensions");
extensions.push_back("minifi-pcap");
extensions.push_back("minifi-rdkafka-extensions");
extensions.push_back("minifi-script-extensions");
extensions.push_back("minifi-usb-camera-extensions");
}
return extensions;
}
};
} /* namespace minifi */
} /* namespace nifi */
} /* namespace apache */
} /* namespace org */
#endif /* AGENT_BUILD_H */
`
is an example.
---
[GitHub] nifi-minifi-cpp pull request #295: MINFICPP-403: Flow Meta tagging
GitHub user minifirocks opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/295 MINFICPP-403: Flow Meta tagging Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/minifirocks/nifi-minifi-cpp meta_info Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/295.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #295 commit 22f52fb9d225233ba63df04eb8c91ca9af94a4ba Author: Bin Qiu Date: 2018-04-03T15:57:23Z MINFICPP-403: Flow Meta tagging ---
[jira] [Commented] (MINIFICPP-445) Implement escape/unescape CSV functions in expression language
[ https://issues.apache.org/jira/browse/MINIFICPP-445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427417#comment-16427417 ] ASF GitHub Bot commented on MINIFICPP-445: -- Github user achristianson commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/293 @apiri sorry about that. Fixed. > Implement escape/unescape CSV functions in expression language > -- > > Key: MINIFICPP-445 > URL: https://issues.apache.org/jira/browse/MINIFICPP-445 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Andrew Christianson >Assignee: Andrew Christianson >Priority: Major > > * > [escapeCsv|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#escapecsv] > * > [unescapeCsv|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#unescapecsv] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp issue #293: MINIFICPP-445 Added escape/unescape CSV expressi...
Github user achristianson commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/293 @apiri sorry about that. Fixed. ---
[jira] [Commented] (NIFI-5045) PutHiveQL routes parse errors to retry instead of failure
[ https://issues.apache.org/jira/browse/NIFI-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427408#comment-16427408 ] ASF GitHub Bot commented on NIFI-5045: -- Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2608 Will review... > PutHiveQL routes parse errors to retry instead of failure > - > > Key: NIFI-5045 > URL: https://issues.apache.org/jira/browse/NIFI-5045 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Minor > > When the input to PutHiveQL is an invalid HiveQL statement, the documentation > for the failure relationship should apply: "A FlowFile is routed to this > relationship if the database cannot be updated and retrying the operation > will also fail, such as an invalid query or an integrity constraint > violation". However, upon invalid HiveQL, the processor routes the flowfile > to 'retry', which is neither desired or logical (the query is bad, how could > it ever succeed on retry?) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2608: NIFI-5045: Fixed error code handling in PutHiveQL
Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2608 Will review... ---
[GitHub] nifi issue #2606: NIFI-5043: TailFile in Multifile mode should not open new ...
Github user mgaido91 commented on the issue: https://github.com/apache/nifi/pull/2606 Thank you for taking a look at this and reviewing it, @markap14! ---
[jira] [Commented] (NIFI-5043) TailFile opens and never closes readers in Multiple Files mode
[
https://issues.apache.org/jira/browse/NIFI-5043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427382#comment-16427382
]
ASF GitHub Bot commented on NIFI-5043:
--
Github user mgaido91 commented on the issue:
https://github.com/apache/nifi/pull/2606
Thank you for taking a look at this and reviewing it, @markap14!
> TailFile opens and never closes readers in Multiple Files mode
> --
>
> Key: NIFI-5043
> URL: https://issues.apache.org/jira/browse/NIFI-5043
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Marco Gaido
>Assignee: Marco Gaido
>Priority: Critical
> Fix For: 1.7.0
>
>
> When Multiple File is set as Tailing Mode, TailFile tries to recover the
> state every time it is triggered, as if it were recovering from a restart. In
> this way, it opens a new reader every time for every file and it never closes
> them.
> This situation can easily be diagnosticated running {{lsof}}: the number of
> open handles keep increasing for every TailFile trigger.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5043) TailFile opens and never closes readers in Multiple Files mode
[
https://issues.apache.org/jira/browse/NIFI-5043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427349#comment-16427349
]
ASF GitHub Bot commented on NIFI-5043:
--
Github user markap14 commented on the issue:
https://github.com/apache/nifi/pull/2606
@mgaido91 thanks for addressing this! And the JIRA did a good job of
explaining exactly what the issue is, as well as how to detect that the issue
was present. Was able to detect the issue on master, then running the PR was
able to ensure that the issue is no longer present. Otherwise, the code looks
good and the changes make sense. +1 merged to master. Thanks again!
> TailFile opens and never closes readers in Multiple Files mode
> --
>
> Key: NIFI-5043
> URL: https://issues.apache.org/jira/browse/NIFI-5043
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Marco Gaido
>Assignee: Marco Gaido
>Priority: Critical
> Fix For: 1.7.0
>
>
> When Multiple File is set as Tailing Mode, TailFile tries to recover the
> state every time it is triggered, as if it were recovering from a restart. In
> this way, it opens a new reader every time for every file and it never closes
> them.
> This situation can easily be diagnosticated running {{lsof}}: the number of
> open handles keep increasing for every TailFile trigger.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (NIFI-5043) TailFile opens and never closes readers in Multiple Files mode
[
https://issues.apache.org/jira/browse/NIFI-5043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne resolved NIFI-5043.
--
Resolution: Fixed
Fix Version/s: 1.7.0
> TailFile opens and never closes readers in Multiple Files mode
> --
>
> Key: NIFI-5043
> URL: https://issues.apache.org/jira/browse/NIFI-5043
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Marco Gaido
>Assignee: Marco Gaido
>Priority: Critical
> Fix For: 1.7.0
>
>
> When Multiple File is set as Tailing Mode, TailFile tries to recover the
> state every time it is triggered, as if it were recovering from a restart. In
> this way, it opens a new reader every time for every file and it never closes
> them.
> This situation can easily be diagnosticated running {{lsof}}: the number of
> open handles keep increasing for every TailFile trigger.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi issue #2606: NIFI-5043: TailFile in Multifile mode should not open new ...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/2606 @mgaido91 thanks for addressing this! And the JIRA did a good job of explaining exactly what the issue is, as well as how to detect that the issue was present. Was able to detect the issue on master, then running the PR was able to ensure that the issue is no longer present. Otherwise, the code looks good and the changes make sense. +1 merged to master. Thanks again! ---
[jira] [Commented] (NIFI-5043) TailFile opens and never closes readers in Multiple Files mode
[
https://issues.apache.org/jira/browse/NIFI-5043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427336#comment-16427336
]
ASF subversion and git services commented on NIFI-5043:
---
Commit c119547222514cf09dd457f0f21b54b307f5f56d in nifi's branch
refs/heads/master from [~mgaido]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=c119547 ]
NIFI-5043: TailFile in Multifile mode should not open new readers in onTrigger
This closes #2606.
Signed-off-by: Mark Payne
> TailFile opens and never closes readers in Multiple Files mode
> --
>
> Key: NIFI-5043
> URL: https://issues.apache.org/jira/browse/NIFI-5043
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Marco Gaido
>Assignee: Marco Gaido
>Priority: Critical
>
> When Multiple File is set as Tailing Mode, TailFile tries to recover the
> state every time it is triggered, as if it were recovering from a restart. In
> this way, it opens a new reader every time for every file and it never closes
> them.
> This situation can easily be diagnosticated running {{lsof}}: the number of
> open handles keep increasing for every TailFile trigger.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5043) TailFile opens and never closes readers in Multiple Files mode
[
https://issues.apache.org/jira/browse/NIFI-5043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427337#comment-16427337
]
ASF GitHub Bot commented on NIFI-5043:
--
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/2606
> TailFile opens and never closes readers in Multiple Files mode
> --
>
> Key: NIFI-5043
> URL: https://issues.apache.org/jira/browse/NIFI-5043
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Marco Gaido
>Assignee: Marco Gaido
>Priority: Critical
>
> When Multiple File is set as Tailing Mode, TailFile tries to recover the
> state every time it is triggered, as if it were recovering from a restart. In
> this way, it opens a new reader every time for every file and it never closes
> them.
> This situation can easily be diagnosticated running {{lsof}}: the number of
> open handles keep increasing for every TailFile trigger.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2606: NIFI-5043: TailFile in Multifile mode should not op...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2606 ---
[jira] [Commented] (NIFI-3599) Add nifi.properties value to globally set the default backpressure size threshold for each connection
[ https://issues.apache.org/jira/browse/NIFI-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427331#comment-16427331 ] ASF GitHub Bot commented on NIFI-3599: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2497 > Add nifi.properties value to globally set the default backpressure size > threshold for each connection > - > > Key: NIFI-3599 > URL: https://issues.apache.org/jira/browse/NIFI-3599 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration >Reporter: Jeremy Dyer >Assignee: Michael Moser >Priority: Major > Fix For: 1.7.0 > > > By default each new connection added to the workflow canvas will have a > default backpressure size threshold of 10,000 objects. While the threshold > can be changed on a connection level it would be convenient to have a global > mechanism for setting that value to something other than 10,000. This > enhancement would add a property to nifi.properties that would allow for this > threshold to be set globally unless otherwise overridden at the connection > level. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-3599) Add nifi.properties value to globally set the default backpressure size threshold for each connection
[ https://issues.apache.org/jira/browse/NIFI-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427333#comment-16427333 ] ASF GitHub Bot commented on NIFI-3599: -- Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2497 @mosermw Looks great! Thanks, this has been merged to master. > Add nifi.properties value to globally set the default backpressure size > threshold for each connection > - > > Key: NIFI-3599 > URL: https://issues.apache.org/jira/browse/NIFI-3599 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration >Reporter: Jeremy Dyer >Assignee: Michael Moser >Priority: Major > Fix For: 1.7.0 > > > By default each new connection added to the workflow canvas will have a > default backpressure size threshold of 10,000 objects. While the threshold > can be changed on a connection level it would be convenient to have a global > mechanism for setting that value to something other than 10,000. This > enhancement would add a property to nifi.properties that would allow for this > threshold to be set globally unless otherwise overridden at the connection > level. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2497: NIFI-3599 Allowed back pressure object count and data size...
Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2497 @mosermw Looks great! Thanks, this has been merged to master. ---
[jira] [Updated] (NIFI-3599) Add nifi.properties value to globally set the default backpressure size threshold for each connection
[ https://issues.apache.org/jira/browse/NIFI-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-3599: -- Resolution: Fixed Fix Version/s: 1.7.0 Status: Resolved (was: Patch Available) > Add nifi.properties value to globally set the default backpressure size > threshold for each connection > - > > Key: NIFI-3599 > URL: https://issues.apache.org/jira/browse/NIFI-3599 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration >Reporter: Jeremy Dyer >Assignee: Michael Moser >Priority: Major > Fix For: 1.7.0 > > > By default each new connection added to the workflow canvas will have a > default backpressure size threshold of 10,000 objects. While the threshold > can be changed on a connection level it would be convenient to have a global > mechanism for setting that value to something other than 10,000. This > enhancement would add a property to nifi.properties that would allow for this > threshold to be set globally unless otherwise overridden at the connection > level. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2497: NIFI-3599 Allowed back pressure object count and da...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2497 ---
[jira] [Commented] (NIFI-3599) Add nifi.properties value to globally set the default backpressure size threshold for each connection
[ https://issues.apache.org/jira/browse/NIFI-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427329#comment-16427329 ] ASF subversion and git services commented on NIFI-3599: --- Commit dc9b4cb516da346a2942d0bef2810b60ac179cc7 in nifi's branch refs/heads/master from [~boardm26] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=dc9b4cb ] NIFI-3599 Allow back pressure object count and data size to be configurable in nifi.properties. This closes #2497 > Add nifi.properties value to globally set the default backpressure size > threshold for each connection > - > > Key: NIFI-3599 > URL: https://issues.apache.org/jira/browse/NIFI-3599 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration >Reporter: Jeremy Dyer >Assignee: Michael Moser >Priority: Major > > By default each new connection added to the workflow canvas will have a > default backpressure size threshold of 10,000 objects. While the threshold > can be changed on a connection level it would be convenient to have a global > mechanism for setting that value to something other than 10,000. This > enhancement would add a property to nifi.properties that would allow for this > threshold to be set globally unless otherwise overridden at the connection > level. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-1295) Add UI option to interrupt a running processor
[ https://issues.apache.org/jira/browse/NIFI-1295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427325#comment-16427325 ] ASF GitHub Bot commented on NIFI-1295: -- Github user moranr commented on the issue: https://github.com/apache/nifi/pull/2607 @mcgilman , I was thinking about some suggestions to the formatting and tooltip labeling used on these. I think we're using the '/' (forward slash) inconsistently which could be a bit confusing. How about formatting the thread count in the global status bar like so: **3 (1)** And the tooltip could read something like: **Threads: Active (Terminated)** Then when the thread icon shows on a processor, the count would again show as '# (#)' and the tooltip could be something like: **3 active threads (1 terminated)** I'm also thinking change the current queue/data size global status format to: **_Object Count (Data Size)_** i.e., use parentheses instead of the forward slash This is consistent with the In & Out stats on components. And one last thing I thought is adding the term 'threads' to the Terminate context menu action. So, change _Terminate_ to _Terminate threads_ > Add UI option to interrupt a running processor > -- > > Key: NIFI-1295 > URL: https://issues.apache.org/jira/browse/NIFI-1295 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Affects Versions: 0.4.0 >Reporter: Oleg Zhurakousky >Assignee: Matt Gilman >Priority: Major > > Basically we need an expose option to a user to kill Processors that can't be > shut down the usual way (see NIFI-78 for more details). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2607: NIFI-1295: Adding UI controls for terminating threads
Github user moranr commented on the issue: https://github.com/apache/nifi/pull/2607 @mcgilman , I was thinking about some suggestions to the formatting and tooltip labeling used on these. I think we're using the '/' (forward slash) inconsistently which could be a bit confusing. How about formatting the thread count in the global status bar like so: **3 (1)** And the tooltip could read something like: **Threads: Active (Terminated)** Then when the thread icon shows on a processor, the count would again show as '# (#)' and the tooltip could be something like: **3 active threads (1 terminated)** I'm also thinking change the current queue/data size global status format to: **_Object Count (Data Size)_** i.e., use parentheses instead of the forward slash This is consistent with the In & Out stats on components. And one last thing I thought is adding the term 'threads' to the Terminate context menu action. So, change _Terminate_ to _Terminate threads_ ---
[jira] [Commented] (MINIFICPP-448) Allow uuid to be built & statically-linked
[ https://issues.apache.org/jira/browse/MINIFICPP-448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427310#comment-16427310 ] ASF GitHub Bot commented on MINIFICPP-448: -- Github user achristianson commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/294 @apiri done. > Allow uuid to be built & statically-linked > -- > > Key: MINIFICPP-448 > URL: https://issues.apache.org/jira/browse/MINIFICPP-448 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Andrew Christianson >Assignee: Andrew Christianson >Priority: Major > > We already bundle uuid code in nifi-minifi-cpp, but there is no way to build > it. The CMakeLists.txt should be updated to allow building of this bundled > uuid and statically-linking it, making it easier to run in environments where > uuid is not available. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp issue #294: MINIFICPP-448 Add uuid external/static link proj...
Github user achristianson commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/294 @apiri done. ---
[jira] [Commented] (NIFI-5044) SelectHiveQL accept only one statement
[
https://issues.apache.org/jira/browse/NIFI-5044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427299#comment-16427299
]
Pierre Villard commented on NIFI-5044:
--
IMO, supporting user defined properties (with EL support) for "set X=Y" would
make a lot of sense and would be really useful.
> SelectHiveQL accept only one statement
> --
>
> Key: NIFI-5044
> URL: https://issues.apache.org/jira/browse/NIFI-5044
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.2.0
>Reporter: Davide Isoardi
>Priority: Critical
>
> In [this
> |[https://github.com/apache/nifi/commit/bbc714e73ba245de7bc32fd9958667c847101f7d]
> ] commit claims to add support to running multiple statements both on
> SelectHiveQL and PutHiveQL; instead, it adds only the support to PutHiveQL,
> so SelectHiveQL still lacks this important feature. @Matt Burgess, I saw that
> you worked on that, is there any reason for this? If not, can we support it?
> If I try to execute this query:
> {quote}set hive.vectorized.execution.enabled = false; SELECT * FROM table_name
> {quote}
> I have this error:
>
> {quote}2018-04-05 13:35:40,572 ERROR [Timer-Driven Process Thread-146]
> o.a.nifi.processors.hive.SelectHiveQL
> SelectHiveQL[id=243d4c17-b1fe-14af--ee8ce15e] Unable to execute
> HiveQL select query set hive.vectorized.execution.enabled = false; SELECT *
> FROM table_name for
> StandardFlowFileRecord[uuid=0e035558-07ce-473b-b0d4-ac00b8b1df93,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1522824912161-2753,
> container=default, section=705], offset=838441,
> length=25],offset=0,name=cliente_attributi.csv,size=25] due to
> org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
> The query did not generate a result set!; routing to failure: {}
> org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
> The query did not generate a result set!
> at
> org.apache.nifi.processors.hive.SelectHiveQL$2.process(SelectHiveQL.java:305)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2529)
> at
> org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:275)
> at
> org.apache.nifi.processors.hive.SelectHiveQL.lambda$onTrigger$0(SelectHiveQL.java:215)
> at
> org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:114)
> at
> org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:106)
> at
> org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:215)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.sql.SQLException: The query did not generate a result set!
> at org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:438)
> at
> org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
> at
> org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
> at
> org.apache.nifi.processors.hive.SelectHiveQL$2.process(SelectHiveQL.java:293)
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5044) SelectHiveQL accept only one statement
[
https://issues.apache.org/jira/browse/NIFI-5044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427226#comment-16427226
]
Matt Burgess commented on NIFI-5044:
The comments only mention some improvements to both processors, the Statement
Delimiter property is only added to (and supported by) PutHiveQL. Changing this
Jira to an Improvement to support multiple statements in SelectHiveQL.
This can be useful for cases such as your example when you need to set
properties on the session and then issue a query, but how would we handle
multiple statements that each return result sets? Would we process each
statement "normally" in turn, issuing flow files that would presumably have
very different schemas?
If we only need to support the "set X=Y" use case, then perhaps we could offer
user-defined properties in SelectHiveQL, and we would use the key value pairs
to issue "set X=Y" commands in the session on the user's behalf, before
executing the query. What do you think [~disoardi]?
> SelectHiveQL accept only one statement
> --
>
> Key: NIFI-5044
> URL: https://issues.apache.org/jira/browse/NIFI-5044
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.2.0
>Reporter: Davide Isoardi
>Priority: Critical
>
> In [this
> |[https://github.com/apache/nifi/commit/bbc714e73ba245de7bc32fd9958667c847101f7d]
> ] commit claims to add support to running multiple statements both on
> SelectHiveQL and PutHiveQL; instead, it adds only the support to PutHiveQL,
> so SelectHiveQL still lacks this important feature. @Matt Burgess, I saw that
> you worked on that, is there any reason for this? If not, can we support it?
> If I try to execute this query:
> {quote}set hive.vectorized.execution.enabled = false; SELECT * FROM table_name
> {quote}
> I have this error:
>
> {quote}2018-04-05 13:35:40,572 ERROR [Timer-Driven Process Thread-146]
> o.a.nifi.processors.hive.SelectHiveQL
> SelectHiveQL[id=243d4c17-b1fe-14af--ee8ce15e] Unable to execute
> HiveQL select query set hive.vectorized.execution.enabled = false; SELECT *
> FROM table_name for
> StandardFlowFileRecord[uuid=0e035558-07ce-473b-b0d4-ac00b8b1df93,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1522824912161-2753,
> container=default, section=705], offset=838441,
> length=25],offset=0,name=cliente_attributi.csv,size=25] due to
> org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
> The query did not generate a result set!; routing to failure: {}
> org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
> The query did not generate a result set!
> at
> org.apache.nifi.processors.hive.SelectHiveQL$2.process(SelectHiveQL.java:305)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2529)
> at
> org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:275)
> at
> org.apache.nifi.processors.hive.SelectHiveQL.lambda$onTrigger$0(SelectHiveQL.java:215)
> at
> org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:114)
> at
> org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:106)
> at
> org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:215)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.sql.SQLException: The query did not generate a result set!
> at org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:438)
> at
> org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
> at
> org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
> at
> org.apache.nifi.processors.hive.SelectHiveQL$2.process(SelectHive
[jira] [Updated] (NIFI-5044) SelectHiveQL accept only one statement
[
https://issues.apache.org/jira/browse/NIFI-5044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated NIFI-5044:
---
Issue Type: Improvement (was: Bug)
> SelectHiveQL accept only one statement
> --
>
> Key: NIFI-5044
> URL: https://issues.apache.org/jira/browse/NIFI-5044
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.2.0
>Reporter: Davide Isoardi
>Priority: Critical
>
> In [this
> |[https://github.com/apache/nifi/commit/bbc714e73ba245de7bc32fd9958667c847101f7d]
> ] commit claims to add support to running multiple statements both on
> SelectHiveQL and PutHiveQL; instead, it adds only the support to PutHiveQL,
> so SelectHiveQL still lacks this important feature. @Matt Burgess, I saw that
> you worked on that, is there any reason for this? If not, can we support it?
> If I try to execute this query:
> {quote}set hive.vectorized.execution.enabled = false; SELECT * FROM table_name
> {quote}
> I have this error:
>
> {quote}2018-04-05 13:35:40,572 ERROR [Timer-Driven Process Thread-146]
> o.a.nifi.processors.hive.SelectHiveQL
> SelectHiveQL[id=243d4c17-b1fe-14af--ee8ce15e] Unable to execute
> HiveQL select query set hive.vectorized.execution.enabled = false; SELECT *
> FROM table_name for
> StandardFlowFileRecord[uuid=0e035558-07ce-473b-b0d4-ac00b8b1df93,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1522824912161-2753,
> container=default, section=705], offset=838441,
> length=25],offset=0,name=cliente_attributi.csv,size=25] due to
> org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
> The query did not generate a result set!; routing to failure: {}
> org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
> The query did not generate a result set!
> at
> org.apache.nifi.processors.hive.SelectHiveQL$2.process(SelectHiveQL.java:305)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2529)
> at
> org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:275)
> at
> org.apache.nifi.processors.hive.SelectHiveQL.lambda$onTrigger$0(SelectHiveQL.java:215)
> at
> org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:114)
> at
> org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:106)
> at
> org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:215)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.sql.SQLException: The query did not generate a result set!
> at org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:438)
> at
> org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
> at
> org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
> at
> org.apache.nifi.processors.hive.SelectHiveQL$2.process(SelectHiveQL.java:293)
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-5045) PutHiveQL routes parse errors to retry instead of failure
[ https://issues.apache.org/jira/browse/NIFI-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-5045: --- Status: Patch Available (was: In Progress) > PutHiveQL routes parse errors to retry instead of failure > - > > Key: NIFI-5045 > URL: https://issues.apache.org/jira/browse/NIFI-5045 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Minor > > When the input to PutHiveQL is an invalid HiveQL statement, the documentation > for the failure relationship should apply: "A FlowFile is routed to this > relationship if the database cannot be updated and retrying the operation > will also fail, such as an invalid query or an integrity constraint > violation". However, upon invalid HiveQL, the processor routes the flowfile > to 'retry', which is neither desired or logical (the query is bad, how could > it ever succeed on retry?) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5045) PutHiveQL routes parse errors to retry instead of failure
[ https://issues.apache.org/jira/browse/NIFI-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427212#comment-16427212 ] ASF GitHub Bot commented on NIFI-5045: -- GitHub user mattyb149 opened a pull request: https://github.com/apache/nifi/pull/2608 NIFI-5045: Fixed error code handling in PutHiveQL Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mattyb149/nifi NIFI-5045 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2608.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2608 commit dd124ad06ca82c5b84a89813a928f49642236a39 Author: Matthew Burgess Date: 2018-04-05T16:28:38Z NIFI-5045: Fixed error code handling in PutHiveQL > PutHiveQL routes parse errors to retry instead of failure > - > > Key: NIFI-5045 > URL: https://issues.apache.org/jira/browse/NIFI-5045 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Minor > > When the input to PutHiveQL is an invalid HiveQL statement, the documentation > for the failure relationship should apply: "A FlowFile is routed to this > relationship if the database cannot be updated and retrying the operation > will also fail, such as an invalid query or an integrity constraint > violation". However, upon invalid HiveQL, the processor routes the flowfile > to 'retry', which is neither desired or logical (the query is bad, how could > it ever succeed on retry?) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2608: NIFI-5045: Fixed error code handling in PutHiveQL
GitHub user mattyb149 opened a pull request: https://github.com/apache/nifi/pull/2608 NIFI-5045: Fixed error code handling in PutHiveQL Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mattyb149/nifi NIFI-5045 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2608.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2608 commit dd124ad06ca82c5b84a89813a928f49642236a39 Author: Matthew Burgess Date: 2018-04-05T16:28:38Z NIFI-5045: Fixed error code handling in PutHiveQL ---
[jira] [Assigned] (NIFI-5045) PutHiveQL routes parse errors to retry instead of failure
[ https://issues.apache.org/jira/browse/NIFI-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess reassigned NIFI-5045: -- Assignee: Matt Burgess > PutHiveQL routes parse errors to retry instead of failure > - > > Key: NIFI-5045 > URL: https://issues.apache.org/jira/browse/NIFI-5045 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Minor > > When the input to PutHiveQL is an invalid HiveQL statement, the documentation > for the failure relationship should apply: "A FlowFile is routed to this > relationship if the database cannot be updated and retrying the operation > will also fail, such as an invalid query or an integrity constraint > violation". However, upon invalid HiveQL, the processor routes the flowfile > to 'retry', which is neither desired or logical (the query is bad, how could > it ever succeed on retry?) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5045) PutHiveQL routes parse errors to retry instead of failure
[ https://issues.apache.org/jira/browse/NIFI-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427166#comment-16427166 ] Matt Burgess commented on NIFI-5045: The current code expects a SQLNonTransientException be thrown on parse errors, and the unit tests use Derby not Hive, the former correctly throws a SQLNonTransientException on parse error. Also unfortunately, the ParseException is not available to the client code. However the SQLException's "vendor code" field is populated by Hive using the following guidelines: 1 to 1: Errors occurring during semantic analysis and compilation of the query. 2 to 2: Runtime errors where Hive believes that retries are unlikely to succeed. 3 to 3: Runtime errors which Hive thinks may be transient and retrying may succeed. 4 to 4: Errors where Hive is unable to advise about retries. Unfortunately, there is no specific error code for parse errors, so the generic error code 4 is returned. However according to the above doc, we can route the flow files to retry or failure based on that guidance. In our case (4), since Hive is unable to advise about retries, we would route the flow files to failure. This logic may change the existing behavior for other errors as well, but I think that's a good thing because we'd be adhering to the error code spec, rather than making our own best guess. > PutHiveQL routes parse errors to retry instead of failure > - > > Key: NIFI-5045 > URL: https://issues.apache.org/jira/browse/NIFI-5045 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Priority: Minor > > When the input to PutHiveQL is an invalid HiveQL statement, the documentation > for the failure relationship should apply: "A FlowFile is routed to this > relationship if the database cannot be updated and retrying the operation > will also fail, such as an invalid query or an integrity constraint > violation". However, upon invalid HiveQL, the processor routes the flowfile > to 'retry', which is neither desired or logical (the query is bad, how could > it ever succeed on retry?) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-5045) PutHiveQL routes parse errors to retry instead of failure
Matt Burgess created NIFI-5045: -- Summary: PutHiveQL routes parse errors to retry instead of failure Key: NIFI-5045 URL: https://issues.apache.org/jira/browse/NIFI-5045 Project: Apache NiFi Issue Type: Bug Components: Extensions Reporter: Matt Burgess When the input to PutHiveQL is an invalid HiveQL statement, the documentation for the failure relationship should apply: "A FlowFile is routed to this relationship if the database cannot be updated and retrying the operation will also fail, such as an invalid query or an integrity constraint violation". However, upon invalid HiveQL, the processor routes the flowfile to 'retry', which is neither desired or logical (the query is bad, how could it ever succeed on retry?) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-5044) SelectHiveQL accept only one statement
[
https://issues.apache.org/jira/browse/NIFI-5044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Isoardi updated NIFI-5044:
-
Description:
In [this
|[https://github.com/apache/nifi/commit/bbc714e73ba245de7bc32fd9958667c847101f7d]
] commit claims to add support to running multiple statements both on
SelectHiveQL and PutHiveQL; instead, it adds only the support to PutHiveQL, so
SelectHiveQL still lacks this important feature. @Matt Burgess, I saw that you
worked on that, is there any reason for this? If not, can we support it?
If I try to execute this query:
{quote}set hive.vectorized.execution.enabled = false; SELECT * FROM table_name
{quote}
I have this error:
{quote}2018-04-05 13:35:40,572 ERROR [Timer-Driven Process Thread-146]
o.a.nifi.processors.hive.SelectHiveQL
SelectHiveQL[id=243d4c17-b1fe-14af--ee8ce15e] Unable to execute HiveQL
select query set hive.vectorized.execution.enabled = false; SELECT * FROM
table_name for
StandardFlowFileRecord[uuid=0e035558-07ce-473b-b0d4-ac00b8b1df93,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1522824912161-2753, container=default,
section=705], offset=838441,
length=25],offset=0,name=cliente_attributi.csv,size=25] due to
org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
The query did not generate a result set!; routing to failure: {}
org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
The query did not generate a result set!
at
org.apache.nifi.processors.hive.SelectHiveQL$2.process(SelectHiveQL.java:305)
at
org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2529)
at
org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:275)
at
org.apache.nifi.processors.hive.SelectHiveQL.lambda$onTrigger$0(SelectHiveQL.java:215)
at
org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:114)
at
org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:106)
at
org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:215)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.sql.SQLException: The query did not generate a result set!
at org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:438)
at
org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
at
org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
at
org.apache.nifi.processors.hive.SelectHiveQL$2.process(SelectHiveQL.java:293)
{quote}
was:
In [this
|[https://github.com/apache/nifi/commit/bbc714e73ba245de7bc32fd9958667c847101f7d]
] commit, of [~mattyb149], it has been added the support for multiple
statements in a script.
If I try to execute this query:
{quote}set hive.vectorized.execution.enabled = false; SELECT * FROM table_name
{quote}
I have this error:
2018-04-05 13:35:40,572 ERROR [Timer-Driven Process Thread-146]
o.a.nifi.processors.hive.SelectHiveQL
SelectHiveQL[id=243d4c17-b1fe-14af--ee8ce15e] Unable to execute HiveQL
select query set hive.vectorized.execution.enabled = false; SELECT * FROM
table_name for
StandardFlowFileRecord[uuid=0e035558-07ce-473b-b0d4-ac00b8b1df93,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1522824912161-2753, container=default,
section=705], offset=838441,
length=25],offset=0,name=cliente_attributi.csv,size=25] due to
org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
The query did not generate a result set!; routing to failure: {}
org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
The query did not generate a result set!
at
org.apache.nifi.processors.hive.SelectHiveQL$2.process(SelectHiveQL.java:305)
at
org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2529)
at
org.
[jira] [Updated] (NIFI-1295) Add UI option to interrupt a running processor
[ https://issues.apache.org/jira/browse/NIFI-1295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-1295: -- Status: Patch Available (was: In Progress) > Add UI option to interrupt a running processor > -- > > Key: NIFI-1295 > URL: https://issues.apache.org/jira/browse/NIFI-1295 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Affects Versions: 0.4.0 >Reporter: Oleg Zhurakousky >Assignee: Matt Gilman >Priority: Major > > Basically we need an expose option to a user to kill Processors that can't be > shut down the usual way (see NIFI-78 for more details). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2607: NIFI-1295: Adding UI controls for terminating threa...
GitHub user mcgilman opened a pull request: https://github.com/apache/nifi/pull/2607 NIFI-1295: Adding UI controls for terminating threads NIFI-1295: - Adding UI controls for terminating hung threads. - Showing current number of terminated threads. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mcgilman/nifi NIFI-1295 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2607.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2607 commit 0c9d34ce8968419f0e950b781d2309fb661f1f2d Author: Matt Gilman Date: 2018-04-05T14:56:54Z NIFI-1295: - Adding UI controls for terminating hung threads. - Showing current number of terminated threads. ---
[jira] [Commented] (NIFI-1295) Add UI option to interrupt a running processor
[ https://issues.apache.org/jira/browse/NIFI-1295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427065#comment-16427065 ] ASF GitHub Bot commented on NIFI-1295: -- GitHub user mcgilman opened a pull request: https://github.com/apache/nifi/pull/2607 NIFI-1295: Adding UI controls for terminating threads NIFI-1295: - Adding UI controls for terminating hung threads. - Showing current number of terminated threads. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mcgilman/nifi NIFI-1295 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2607.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2607 commit 0c9d34ce8968419f0e950b781d2309fb661f1f2d Author: Matt Gilman Date: 2018-04-05T14:56:54Z NIFI-1295: - Adding UI controls for terminating hung threads. - Showing current number of terminated threads. > Add UI option to interrupt a running processor > -- > > Key: NIFI-1295 > URL: https://issues.apache.org/jira/browse/NIFI-1295 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Affects Versions: 0.4.0 >Reporter: Oleg Zhurakousky >Assignee: Matt Gilman >Priority: Major > > Basically we need an expose option to a user to kill Processors that can't be > shut down the usual way (see NIFI-78 for more details). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-5044) SelectHiveQL accept only one statement
Davide Isoardi created NIFI-5044:
Summary: SelectHiveQL accept only one statement
Key: NIFI-5044
URL: https://issues.apache.org/jira/browse/NIFI-5044
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 1.2.0
Reporter: Davide Isoardi
In [this
|[https://github.com/apache/nifi/commit/bbc714e73ba245de7bc32fd9958667c847101f7d]]
commit, of [~mattyb149], it has been added the support for multiple statements
in a script.
If I try to execute this query:
{quote}set hive.vectorized.execution.enabled = false; SELECT * FROM table_name
{quote}
I have this error:
2018-04-05 13:35:40,572 ERROR [Timer-Driven Process Thread-146]
o.a.nifi.processors.hive.SelectHiveQL
SelectHiveQL[id=243d4c17-b1fe-14af--ee8ce15e] Unable to execute HiveQL
select query set hive.vectorized.execution.enabled = false; SELECT * FROM
table_name for
StandardFlowFileRecord[uuid=0e035558-07ce-473b-b0d4-ac00b8b1df93,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1522824912161-2753, container=default,
section=705], offset=838441,
length=25],offset=0,name=cliente_attributi.csv,size=25] due to
org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
The query did not generate a result set!; routing to failure: {}
org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
The query did not generate a result set!
at
org.apache.nifi.processors.hive.SelectHiveQL$2.process(SelectHiveQL.java:305)
at
org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2529)
at
org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:275)
at
org.apache.nifi.processors.hive.SelectHiveQL.lambda$onTrigger$0(SelectHiveQL.java:215)
at
org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:114)
at
org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:106)
at
org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:215)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.sql.SQLException: The query did not generate a result set!
at org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:438)
at
org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
at
org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
at
org.apache.nifi.processors.hive.SelectHiveQL$2.process(SelectHiveQL.java:293)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-5044) SelectHiveQL accept only one statement
[
https://issues.apache.org/jira/browse/NIFI-5044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Isoardi updated NIFI-5044:
-
Description:
In [this
|[https://github.com/apache/nifi/commit/bbc714e73ba245de7bc32fd9958667c847101f7d]
] commit, of [~mattyb149], it has been added the support for multiple
statements in a script.
If I try to execute this query:
{quote}set hive.vectorized.execution.enabled = false; SELECT * FROM table_name
{quote}
I have this error:
2018-04-05 13:35:40,572 ERROR [Timer-Driven Process Thread-146]
o.a.nifi.processors.hive.SelectHiveQL
SelectHiveQL[id=243d4c17-b1fe-14af--ee8ce15e] Unable to execute HiveQL
select query set hive.vectorized.execution.enabled = false; SELECT * FROM
table_name for
StandardFlowFileRecord[uuid=0e035558-07ce-473b-b0d4-ac00b8b1df93,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1522824912161-2753, container=default,
section=705], offset=838441,
length=25],offset=0,name=cliente_attributi.csv,size=25] due to
org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
The query did not generate a result set!; routing to failure: {}
org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
The query did not generate a result set!
at
org.apache.nifi.processors.hive.SelectHiveQL$2.process(SelectHiveQL.java:305)
at
org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2529)
at
org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:275)
at
org.apache.nifi.processors.hive.SelectHiveQL.lambda$onTrigger$0(SelectHiveQL.java:215)
at
org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:114)
at
org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:106)
at
org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:215)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.sql.SQLException: The query did not generate a result set!
at org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:438)
at
org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
at
org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
at
org.apache.nifi.processors.hive.SelectHiveQL$2.process(SelectHiveQL.java:293)
was:
In [this
|[https://github.com/apache/nifi/commit/bbc714e73ba245de7bc32fd9958667c847101f7d]]
commit, of [~mattyb149], it has been added the support for multiple statements
in a script.
If I try to execute this query:
{quote}set hive.vectorized.execution.enabled = false; SELECT * FROM table_name
{quote}
I have this error:
2018-04-05 13:35:40,572 ERROR [Timer-Driven Process Thread-146]
o.a.nifi.processors.hive.SelectHiveQL
SelectHiveQL[id=243d4c17-b1fe-14af--ee8ce15e] Unable to execute HiveQL
select query set hive.vectorized.execution.enabled = false; SELECT * FROM
table_name for
StandardFlowFileRecord[uuid=0e035558-07ce-473b-b0d4-ac00b8b1df93,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1522824912161-2753, container=default,
section=705], offset=838441,
length=25],offset=0,name=cliente_attributi.csv,size=25] due to
org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
The query did not generate a result set!; routing to failure: {}
org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
The query did not generate a result set!
at
org.apache.nifi.processors.hive.SelectHiveQL$2.process(SelectHiveQL.java:305)
at
org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2529)
at
org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:275)
at
org.apache.nifi.processors.hive.SelectHiveQL.lambda$onTrigger$0(SelectHiveQL.java:215)
at
org.apache.nifi.processor.util.pattern.PartialFunction
[jira] [Commented] (NIFI-5043) TailFile opens and never closes readers in Multiple Files mode
[
https://issues.apache.org/jira/browse/NIFI-5043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16426802#comment-16426802
]
ASF GitHub Bot commented on NIFI-5043:
--
GitHub user mgaido91 opened a pull request:
https://github.com/apache/nifi/pull/2606
NIFI-5043: TailFile in Multifile mode should not open new readers in
onTrigger
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with NIFI- where is the JIRA number
you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically master)?
- [x] Is your initial contribution a single, squashed commit?
### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
**Manual tests (running lsof while the processor runs)**
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file, including the main
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in
which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mgaido91/nifi NIFI-5043
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/2606.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2606
commit d688846af44f2647d4d3e38508f9af4ca503632f
Author: Marco Gaido
Date: 2018-04-05T11:44:55Z
NIFI-5043: TailFile in Multifile mode should not open new readers in
onTrigger
> TailFile opens and never closes readers in Multiple Files mode
> --
>
> Key: NIFI-5043
> URL: https://issues.apache.org/jira/browse/NIFI-5043
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Marco Gaido
>Assignee: Marco Gaido
>Priority: Critical
>
> When Multiple File is set as Tailing Mode, TailFile tries to recover the
> state every time it is triggered, as if it were recovering from a restart. In
> this way, it opens a new reader every time for every file and it never closes
> them.
> This situation can easily be diagnosticated running {{lsof}}: the number of
> open handles keep increasing for every TailFile trigger.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2606: NIFI-5043: TailFile in Multifile mode should not op...
GitHub user mgaido91 opened a pull request: https://github.com/apache/nifi/pull/2606 NIFI-5043: TailFile in Multifile mode should not open new readers in onTrigger Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? **Manual tests (running lsof while the processor runs)** - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mgaido91/nifi NIFI-5043 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2606.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2606 commit d688846af44f2647d4d3e38508f9af4ca503632f Author: Marco Gaido Date: 2018-04-05T11:44:55Z NIFI-5043: TailFile in Multifile mode should not open new readers in onTrigger ---
[jira] [Created] (NIFI-5043) TailFile opens and never closes readers in Multiple Files mode
Marco Gaido created NIFI-5043:
-
Summary: TailFile opens and never closes readers in Multiple Files
mode
Key: NIFI-5043
URL: https://issues.apache.org/jira/browse/NIFI-5043
Project: Apache NiFi
Issue Type: Bug
Reporter: Marco Gaido
Assignee: Marco Gaido
When Multiple File is set as Tailing Mode, TailFile tries to recover the state
every time it is triggered, as if it were recovering from a restart. In this
way, it opens a new reader every time for every file and it never closes them.
This situation can easily be diagnosticated running {{lsof}}: the number of
open handles keep increasing for every TailFile trigger.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
