[jira] [Commented] (NIFI-4393) AbstractDatabaseFetchProcessor handles SQL Server brackets incorrectly

[Koji Kawamura (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335467#comment-16335467
 ] 

Koji Kawamura commented on NIFI-4393:
-

[PR 2424|https://github.com/apache/nifi/pull/2424] is ready for review.

> AbstractDatabaseFetchProcessor handles SQL Server brackets incorrectly
> --
>
> Key: NIFI-4393
> URL: https://issues.apache.org/jira/browse/NIFI-4393
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.4.0
>Reporter: Mikołaj Siedlarek
>Assignee: Koji Kawamura
>Priority: Major
> Attachments: 
> 0001-Handle-SQL-Server-square-brackets-in-AbstractDatabas.patch
>
>
> SQL Server allows column names to contain whitespace, in which case they are 
> written in SQL queries inside square brackets. When using processors based on 
> {{AbstractDatabaseFetchProcessor}}, such as {{QueryDatabaseTable}} they have 
> to be specified in  "Maximum-value Columns" in square brackets, because 
> otherwise they would break a SELECT query. However, when such column name is 
> retrieved from ResultSetMetaData, the driver returns it without square 
> brackets. This causes a mismatch between the key used to save last seen 
> maximum value in processor's state and the one used to search for the value 
> later.
> I'm not sure whether the attached patch is very elegant, but it fixes the 
> issue in a simplest way possible.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4393) AbstractDatabaseFetchProcessor handles SQL Server brackets incorrectly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335459#comment-16335459
 ] 

ASF GitHub Bot commented on NIFI-4393:
--

GitHub user ijokarumawak opened a pull request:

https://github.com/apache/nifi/pull/2424

NIFI-4393: Handle database specific identifier escape characters

QueryDatabaseTable and GenerateTableFetch processors were not able to
use max value state as expected, if max value column was wrapped with
escape characters. Due to a mis-match between computed state keys
and actual keys used in the managed state. State keys computed by
getStateKey method included escape characters while actual stored keys
did not. Resulted querying the same dataset again and again.

This commit added unwrapIdentifier method to DatabaseAdapter class to
remove database specific escape characters for identifiers such as table
and column names, so that max value state keys are populated correctly
even if identifiers are wrapped with escape characters.

This commit also added new DatabaseAdapter for MySQL, to handle MySQL
specific identifier escape with back-ticks.

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [x] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [x] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [x] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ijokarumawak/nifi nifi-4393

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2424.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2424


commit 93fbb976a246a75aaa6206fdafe68c6ba3ed571a
Author: Koji Kawamura 
Date:   2018-01-23T06:15:36Z

NIFI-4393: Handle database specific identifier escape characters

QueryDatabaseTable and GenerateTableFetch processors were not able to
use max value state as expected, if max value column was wrapped with
escape characters. Due to a mis-match between computed state keys
and actual keys used in the managed state. State keys computed by
getStateKey method included escape characters while actual stored keys
did not. Resulted querying the same dataset again and again.

This commit added unwrapIdentifier method to DatabaseAdapter class to
remove database specific escape characters for identifiers such as table
and column names, so that max value state keys are populated correctly
even if identifiers are wrapped with escape characters.

This commit also added new DatabaseAdapter for MySQL, to handle MySQL
specific identifier escape with back-ticks.




> AbstractDatabaseFetchProcessor handles SQL Server brackets incorrectly
> --
>
> Key: NIFI-4393
> URL: https://issues.apache.org/jira/browse/NIFI-4393
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.4.0
>Reporter: Mikołaj Siedlarek
>Assignee: Koji Kawamura
>Priority: Major
> 

[GitHub] nifi pull request #2424: NIFI-4393: Handle database specific identifier esca...

[ijokarumawak]

GitHub user ijokarumawak opened a pull request:

https://github.com/apache/nifi/pull/2424

NIFI-4393: Handle database specific identifier escape characters

QueryDatabaseTable and GenerateTableFetch processors were not able to
use max value state as expected, if max value column was wrapped with
escape characters. Due to a mis-match between computed state keys
and actual keys used in the managed state. State keys computed by
getStateKey method included escape characters while actual stored keys
did not. Resulted querying the same dataset again and again.

This commit added unwrapIdentifier method to DatabaseAdapter class to
remove database specific escape characters for identifiers such as table
and column names, so that max value state keys are populated correctly
even if identifiers are wrapped with escape characters.

This commit also added new DatabaseAdapter for MySQL, to handle MySQL
specific identifier escape with back-ticks.

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [x] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [x] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [x] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ijokarumawak/nifi nifi-4393

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2424.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2424


commit 93fbb976a246a75aaa6206fdafe68c6ba3ed571a
Author: Koji Kawamura 
Date:   2018-01-23T06:15:36Z

NIFI-4393: Handle database specific identifier escape characters

QueryDatabaseTable and GenerateTableFetch processors were not able to
use max value state as expected, if max value column was wrapped with
escape characters. Due to a mis-match between computed state keys
and actual keys used in the managed state. State keys computed by
getStateKey method included escape characters while actual stored keys
did not. Resulted querying the same dataset again and again.

This commit added unwrapIdentifier method to DatabaseAdapter class to
remove database specific escape characters for identifiers such as table
and column names, so that max value state keys are populated correctly
even if identifiers are wrapped with escape characters.

This commit also added new DatabaseAdapter for MySQL, to handle MySQL
specific identifier escape with back-ticks.




---

[jira] [Commented] (NIFI-4796) template parameter is missing in swagger.json for /process-groups/{id}/templates/upload API call

[JIRA]

[ 
https://issues.apache.org/jira/browse/NIFI-4796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335454#comment-16335454
 ] 

Sébastien Bouchex Bellomié commented on NIFI-4796:
--

It is indeed ta duplicate and yes, the solution of reverting to the previous 
type makes sense. Duplicating the api call is useless considering that the 
original one is not working.

> template parameter is missing in swagger.json for 
> /process-groups/{id}/templates/upload API call
> 
>
> Key: NIFI-4796
> URL: https://issues.apache.org/jira/browse/NIFI-4796
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0
>Reporter: Sébastien Bouchex Bellomié
>Priority: Major
>
> "template" is defined as a File parameter in the 
> /process-groups/\{id}/templates/upload API code and once the swagger.json has 
> been generated, the parameter is missing. Only id is present.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4393) AbstractDatabaseFetchProcessor handles SQL Server brackets incorrectly

[Koji Kawamura (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335410#comment-16335410
 ] 

Koji Kawamura commented on NIFI-4393:
-

Hi [~msiedlarek] [~mattyb149], I came across this issue, but with MySQL in my 
case. MySQL allows table and column names to be wrapped with back-ticks(`). I 
was trying to fix it with similar approach that [~msiedlarek] did in the patch. 
But I agree with what [~mattyb149] suggested. Now I moved the unwrap logic into 
DatabaseAdapters. Testing with various database engines now (MySQL, PostgreSQL 
and Microsoft SQL Server). I will send a PR shortly once I finished the test. 
Thank you!

> AbstractDatabaseFetchProcessor handles SQL Server brackets incorrectly
> --
>
> Key: NIFI-4393
> URL: https://issues.apache.org/jira/browse/NIFI-4393
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.4.0
>Reporter: Mikołaj Siedlarek
>Assignee: Koji Kawamura
>Priority: Major
> Attachments: 
> 0001-Handle-SQL-Server-square-brackets-in-AbstractDatabas.patch
>
>
> SQL Server allows column names to contain whitespace, in which case they are 
> written in SQL queries inside square brackets. When using processors based on 
> {{AbstractDatabaseFetchProcessor}}, such as {{QueryDatabaseTable}} they have 
> to be specified in  "Maximum-value Columns" in square brackets, because 
> otherwise they would break a SELECT query. However, when such column name is 
> retrieved from ResultSetMetaData, the driver returns it without square 
> brackets. This causes a mismatch between the key used to save last seen 
> maximum value in processor's state and the one used to search for the value 
> later.
> I'm not sure whether the attached patch is very elegant, but it fixes the 
> issue in a simplest way possible.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (NIFI-4393) AbstractDatabaseFetchProcessor handles SQL Server brackets incorrectly

[Koji Kawamura (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Koji Kawamura reassigned NIFI-4393:
---

Assignee: Koji Kawamura

> AbstractDatabaseFetchProcessor handles SQL Server brackets incorrectly
> --
>
> Key: NIFI-4393
> URL: https://issues.apache.org/jira/browse/NIFI-4393
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.4.0
>Reporter: Mikołaj Siedlarek
>Assignee: Koji Kawamura
>Priority: Major
> Attachments: 
> 0001-Handle-SQL-Server-square-brackets-in-AbstractDatabas.patch
>
>
> SQL Server allows column names to contain whitespace, in which case they are 
> written in SQL queries inside square brackets. When using processors based on 
> {{AbstractDatabaseFetchProcessor}}, such as {{QueryDatabaseTable}} they have 
> to be specified in  "Maximum-value Columns" in square brackets, because 
> otherwise they would break a SELECT query. However, when such column name is 
> retrieved from ResultSetMetaData, the driver returns it without square 
> brackets. This causes a mismatch between the key used to save last seen 
> maximum value in processor's state and the one used to search for the value 
> later.
> I'm not sure whether the attached patch is very elegant, but it fixes the 
> issue in a simplest way possible.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4801) Rest-api swagger definition produces non-functional template import in Python

[Daniel Chaffelson (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335387#comment-16335387
 ] 

Daniel Chaffelson commented on NIFI-4801:
-

In 1.2/1.3/1.4 we had:
{code:java}
"parameters" : [ {
"name" : "id",
"in" : "path",
"description" : "The process group id.",
"required" : true,
"type" : "string"
}, {
"name" : "template",
"in" : "formData",
"required" : false,
"type" : "file"
} ]
{code}
Would it not make more sense to revert to the previous method than introducing 
a new method? Unless of course this is to bring it into consistency with other 
methods in the platform, it would make more sense to me to revert.

I can easily test the resulting swagger.json, just let me know which version is 
preferred or provide me a sample compile.

> Rest-api swagger definition produces non-functional template import in Python
> -
>
> Key: NIFI-4801
> URL: https://issues.apache.org/jira/browse/NIFI-4801
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0
> Environment: Python 2.7/3.6
>Reporter: Daniel Chaffelson
>Assignee: Kevin Doran
>Priority: Major
>
> The swagger.json produced when compiling NiFi-1.5.0 results in a 
> ProcessgroupsApi().upload_template function that only accepts the id of the 
> Process Group to receive the template, and no option to specify the template 
> itself.
> It would appear that the underlying API call expects the template to be the 
> body of the request, but the produced function does not allow it to be 
> specified. This is changed from NiFi-1.2.0 where a  'template' keyword 
> argument was included.
> It may also be related to how the TemplatesApi().export_template function 
> used to produce a TemplateDTO and now produces a string. 
> I am unsure in which version since 1.2.0 this changed, it may not 
> specifically be just 1.5.0 code.
> An example of the procedurally generated code can be found at:
> [https://github.com/Chaffelson/nifi-python-swagger-client/blob/master/swagger_client/apis/processgroups_api.py]
> And documentation at:
> [http://nifi-python-swagger-client.readthedocs.io/en/latest/ProcessgroupsApi/#upload_template]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (MINIFICPP-377) Resolve test failures as a result of logging changes.

[Aldrin Piri (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aldrin Piri updated MINIFICPP-377:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Resolve test failures as a result of logging changes. 
> --
>
> Key: MINIFICPP-377
> URL: https://issues.apache.org/jira/browse/MINIFICPP-377
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Affects Versions: 0.4.0
>Reporter: marco polo
>Assignee: marco polo
>Priority: Blocker
> Fix For: 0.4.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MINIFICPP-377) Resolve test failures as a result of logging changes.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335331#comment-16335331
 ] 

ASF GitHub Bot commented on MINIFICPP-377:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi-cpp/pull/246


> Resolve test failures as a result of logging changes. 
> --
>
> Key: MINIFICPP-377
> URL: https://issues.apache.org/jira/browse/MINIFICPP-377
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Affects Versions: 0.4.0
>Reporter: marco polo
>Assignee: marco polo
>Priority: Blocker
> Fix For: 0.4.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi-minifi-cpp pull request #246: MINIFICPP-377: Change log level for Invok...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi-cpp/pull/246


---

[jira] [Commented] (MINIFICPP-377) Resolve test failures as a result of logging changes.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335329#comment-16335329
 ] 

ASF GitHub Bot commented on MINIFICPP-377:
--

Github user apiri commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/246
  
drat, definitely overlooked running.  good catch and thanks for fixing.  
build/tests look good and will merge


> Resolve test failures as a result of logging changes. 
> --
>
> Key: MINIFICPP-377
> URL: https://issues.apache.org/jira/browse/MINIFICPP-377
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Affects Versions: 0.4.0
>Reporter: marco polo
>Assignee: marco polo
>Priority: Blocker
> Fix For: 0.4.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi-minifi-cpp issue #246: MINIFICPP-377: Change log level for InvokeHTTP t...

[apiri]

Github user apiri commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/246
  
drat, definitely overlooked running.  good catch and thanks for fixing.  
build/tests look good and will merge


---

[jira] [Updated] (MINIFICPP-378) New threads should detach when battery monitor reduces them

[marco polo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

marco polo updated MINIFICPP-378:
-
Status: Patch Available  (was: Open)

> New threads should detach when battery monitor reduces them
> ---
>
> Key: MINIFICPP-378
> URL: https://issues.apache.org/jira/browse/MINIFICPP-378
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Affects Versions: 0.4.0
>Reporter: marco polo
>Assignee: marco polo
>Priority: Major
> Fix For: 0.4.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-3518) Create a Morphlines processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335144#comment-16335144
 ] 

ASF GitHub Bot commented on NIFI-3518:
--

Github user WilliamNouet commented on the issue:

https://github.com/apache/nifi/pull/2028
  
@joewitt, did you get a chance to review the latest commit?


> Create a Morphlines processor
> -
>
> Key: NIFI-3518
> URL: https://issues.apache.org/jira/browse/NIFI-3518
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: William Nouet
>Priority: Minor
>
> Create a dedicate processor to run Morphlines transformations 
> (http://kitesdk.org/docs/1.1.0/morphlines/morphlines-reference-guide.html) 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi issue #2028: NIFI-3518 Create a Morphlines processor

[WilliamNouet]

Github user WilliamNouet commented on the issue:

https://github.com/apache/nifi/pull/2028
  
@joewitt, did you get a chance to review the latest commit?


---

[GitHub] nifi issue #2101: NIFI-4289 - InfluxDB put processor

[mans2singh]

Github user mans2singh commented on the issue:

https://github.com/apache/nifi/pull/2101
  
@MikeThomsen - 

Thanks for your review comments.

I've updated the version and rebased the code.  

Regarding the error - InfluxDB uses line protocol for writing data (I've 
mentioned that in the capabilities section of the processor) - 
[https://docs.influxdata.com/influxdb/v1.4/write_protocols/line_protocol_tutorial/]
 to for writing data.  

I have integration/unit test that use the protocol for good and bad format 
scenarios -

- integration test - 
[https://github.com/apache/nifi/pull/2101/files#diff-6aa4a2accea7ae7b489e2a219fb777f2]
- unit tests - 
[https://github.com/apache/nifi/pull/2101/files#diff-9f4273a51b147a9e22085ef842726664]

If you are using the flow file generator with the line protocol - the 
timestamp (last field) is a number - so please use `${now():toNumber()}` rather 
than `${now():toString()}` as shown in the sample measurement below:
`water,country=US,city=sf rain=1,humidity=0.6 ${now():toNumber()}`

Please let me know of you have any additional comments/recommendations or 
if there is anything else required.

Thanks


---

[jira] [Commented] (NIFI-4289) Implement put processor for InfluxDB

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335063#comment-16335063
 ] 

ASF GitHub Bot commented on NIFI-4289:
--

Github user mans2singh commented on the issue:

https://github.com/apache/nifi/pull/2101
  
@MikeThomsen - 

Thanks for your review comments.

I've updated the version and rebased the code.  

Regarding the error - InfluxDB uses line protocol for writing data (I've 
mentioned that in the capabilities section of the processor) - 
[https://docs.influxdata.com/influxdb/v1.4/write_protocols/line_protocol_tutorial/]
 to for writing data.  

I have integration/unit test that use the protocol for good and bad format 
scenarios -

- integration test - 
[https://github.com/apache/nifi/pull/2101/files#diff-6aa4a2accea7ae7b489e2a219fb777f2]
- unit tests - 
[https://github.com/apache/nifi/pull/2101/files#diff-9f4273a51b147a9e22085ef842726664]

If you are using the flow file generator with the line protocol - the 
timestamp (last field) is a number - so please use `${now():toNumber()}` rather 
than `${now():toString()}` as shown in the sample measurement below:
`water,country=US,city=sf rain=1,humidity=0.6 ${now():toNumber()}`

Please let me know of you have any additional comments/recommendations or 
if there is anything else required.

Thanks


> Implement put processor for InfluxDB
> 
>
> Key: NIFI-4289
> URL: https://issues.apache.org/jira/browse/NIFI-4289
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Affects Versions: 1.3.0
> Environment: All
>Reporter: Mans Singh
>Assignee: Mans Singh
>Priority: Minor
>  Labels: insert, measurements,, put, timeseries
>
> Support inserting time series measurements into InfluxDB.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4796) template parameter is missing in swagger.json for /process-groups/{id}/templates/upload API call

[Kevin Doran (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334957#comment-16334957
 ] 

Kevin Doran commented on NIFI-4796:
---

[~sbouc...@infovista.com], please see NIFI-4801, which duplicates this ticket. 
The comments there have a potential solution. Let me know on that ticket if (1) 
it is indeed a duplicate of this issue you reported and (2) if the output of my 
proposed solution looks correct to resolve this bug. Thanks!

> template parameter is missing in swagger.json for 
> /process-groups/{id}/templates/upload API call
> 
>
> Key: NIFI-4796
> URL: https://issues.apache.org/jira/browse/NIFI-4796
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0
>Reporter: Sébastien Bouchex Bellomié
>Priority: Major
>
> "template" is defined as a File parameter in the 
> /process-groups/\{id}/templates/upload API code and once the swagger.json has 
> been generated, the parameter is missing. Only id is present.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi issue #2408: localization using the JSTL standard fmt tag for multiling...

[mcgilman]

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/2408
  
@ns7381 I've started reviewing the PR. This is a great addition. Thank you 
for taking the time to post it. Before we merge it in, I have a couple requests.

- The changes appear to cause the icons in the toolbar at the top to become 
disable with a vanilla unsecured instance. I haven't been able to continue 
verifying functionality as I'm blocked on this issue.

- The client-side code is structured in a modular format. Each module has 
any dependent modules passed into it. Rather than directly referencing nf._ on 
the global scope throughout the code base, can you please pass in a reference 
to nf._ in each module that references it? See the top of any 
nf-.js for an example.

- The Messages-.properties are under src/main/java. Can these 
be moved to src/main/resources?

- Some of the keys in resources.js have generic names like Message11. Can 
we use meaningful names throughout?

- I think we may need to introduce some defensive programming in some cases 
now that the values which were previously string literals have been abstracted 
and accessed via a lookup. Since there is no intention to support markup in the 
values, we should not pass them into functions that support interpreting 
markup. Instead that should only be set via `.text(...)` or escaped prior to 
passing in to `.html(...)`, `.append(...)`, etc. You should see other instances 
of this throughout the codebase, but I am happy to help identify specific cases 
as the review progresses.

- What strategies are available for applying localization to messages 
returned from the server? Should a follow-on JIRA be filed to help in this area?

Thanks!


---

[jira] [Commented] (NIFI-4801) Rest-api swagger definition produces non-functional template import in Python

[Matt Gilman (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334891#comment-16334891
 ] 

Matt Gilman commented on NIFI-4801:
---

[~kdoran] The change makes sense to me, but I'll allow someone who is actually 
consuming these APIs through that spec to confirm it's accuracy.

> Rest-api swagger definition produces non-functional template import in Python
> -
>
> Key: NIFI-4801
> URL: https://issues.apache.org/jira/browse/NIFI-4801
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0
> Environment: Python 2.7/3.6
>Reporter: Daniel Chaffelson
>Assignee: Kevin Doran
>Priority: Major
>
> The swagger.json produced when compiling NiFi-1.5.0 results in a 
> ProcessgroupsApi().upload_template function that only accepts the id of the 
> Process Group to receive the template, and no option to specify the template 
> itself.
> It would appear that the underlying API call expects the template to be the 
> body of the request, but the produced function does not allow it to be 
> specified. This is changed from NiFi-1.2.0 where a  'template' keyword 
> argument was included.
> It may also be related to how the TemplatesApi().export_template function 
> used to produce a TemplateDTO and now produces a string. 
> I am unsure in which version since 1.2.0 this changed, it may not 
> specifically be just 1.5.0 code.
> An example of the procedurally generated code can be found at:
> [https://github.com/Chaffelson/nifi-python-swagger-client/blob/master/swagger_client/apis/processgroups_api.py]
> And documentation at:
> [http://nifi-python-swagger-client.readthedocs.io/en/latest/ProcessgroupsApi/#upload_template]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4801) Rest-api swagger definition produces non-functional template import in Python

[Kevin Doran (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334880#comment-16334880
 ] 

Kevin Doran commented on NIFI-4801:
---

[~mcgilman] I added this annotation to 
{{ProcessGroupResource.uploadTemplate(...)}}:

{code}
@ApiImplicitParams(
value = {
@ApiImplicitParam(
name = "body",
value = "The binary content of the template file being uploaded",
required = true,
dataType = "java.io.File",
paramType = "body")
}
)
{code}

which changes the swagger.json output to:

{noformat}
...
"/process-groups/{id}/templates/upload" : {
  "post" : {
"tags" : [ "process-groups" ],
"summary" : "Uploads a template",
"description" : "",
"operationId" : "uploadTemplate",
"consumes" : [ "multipart/form-data" ],
"produces" : [ "application/xml" ],
"parameters" : [ {
  "name" : "id",
  "in" : "path",
  "description" : "The process group id.",
  "required" : true,
  "type" : "string"
}, {
  "in" : "body",
  "name" : "body",
  "description" : "The binary content of the template file being uploaded",
  "required" : true,
  "schema" : {
"type" : "file"
  }
} ],
"responses" : {
  "200" : {
"description" : "successful operation",
"schema" : {
  "$ref" : "#/definitions/TemplateEntity"
}
  },
  "400" : {
"description" : "NiFi was unable to complete the request because it was 
invalid. The request should not be retried without modification."
  },
  "401" : {
"description" : "Client could not be authenticated."
  },
  "403" : {
"description" : "Client is not authorized to make this request."
  },
  "409" : {
"description" : "The request was valid but NiFi was not in the 
appropriate state to process it. Retrying the same request later may be 
successful."
  }
},
"security" : [ {
  "Write - /process-groups/{uuid}" : [ ]
} ]
  }
},
...
{noformat}

[~chaffelson] - does that output look correct to you? any quick way to test it 
will, if used as an input, procedurally generate the correct python code?

> Rest-api swagger definition produces non-functional template import in Python
> -
>
> Key: NIFI-4801
> URL: https://issues.apache.org/jira/browse/NIFI-4801
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0
> Environment: Python 2.7/3.6
>Reporter: Daniel Chaffelson
>Assignee: Kevin Doran
>Priority: Major
>
> The swagger.json produced when compiling NiFi-1.5.0 results in a 
> ProcessgroupsApi().upload_template function that only accepts the id of the 
> Process Group to receive the template, and no option to specify the template 
> itself.
> It would appear that the underlying API call expects the template to be the 
> body of the request, but the produced function does not allow it to be 
> specified. This is changed from NiFi-1.2.0 where a  'template' keyword 
> argument was included.
> It may also be related to how the TemplatesApi().export_template function 
> used to produce a TemplateDTO and now produces a string. 
> I am unsure in which version since 1.2.0 this changed, it may not 
> specifically be just 1.5.0 code.
> An example of the procedurally generated code can be found at:
> [https://github.com/Chaffelson/nifi-python-swagger-client/blob/master/swagger_client/apis/processgroups_api.py]
> And documentation at:
> [http://nifi-python-swagger-client.readthedocs.io/en/latest/ProcessgroupsApi/#upload_template]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (NIFI-4801) Rest-api swagger definition produces non-functional template import in Python

[Kevin Doran (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334880#comment-16334880
 ] 

Kevin Doran edited comment on NIFI-4801 at 1/22/18 8:53 PM:


[~mcgilman] On a branch, I added this annotation to 
{{ProcessGroupResource.uploadTemplate(...)}}:
{code:java}
@ApiImplicitParams(
value = {
@ApiImplicitParam(
name = "body",
value = "The binary content of the template file being uploaded",
required = true,
dataType = "java.io.File",
paramType = "body")
}
)
{code}
which changes the swagger.json output to:
{noformat}
...
"/process-groups/{id}/templates/upload" : {
  "post" : {
"tags" : [ "process-groups" ],
"summary" : "Uploads a template",
"description" : "",
"operationId" : "uploadTemplate",
"consumes" : [ "multipart/form-data" ],
"produces" : [ "application/xml" ],
"parameters" : [ {
  "name" : "id",
  "in" : "path",
  "description" : "The process group id.",
  "required" : true,
  "type" : "string"
}, {
  "in" : "body",
  "name" : "body",
  "description" : "The binary content of the template file being uploaded",
  "required" : true,
  "schema" : {
"type" : "file"
  }
} ],
"responses" : {
  "200" : {
"description" : "successful operation",
"schema" : {
  "$ref" : "#/definitions/TemplateEntity"
}
  },
  "400" : {
"description" : "NiFi was unable to complete the request because it was 
invalid. The request should not be retried without modification."
  },
  "401" : {
"description" : "Client could not be authenticated."
  },
  "403" : {
"description" : "Client is not authorized to make this request."
  },
  "409" : {
"description" : "The request was valid but NiFi was not in the 
appropriate state to process it. Retrying the same request later may be 
successful."
  }
},
"security" : [ {
  "Write - /process-groups/{uuid}" : [ ]
} ]
  }
},
...
{noformat}
[~chaffelson] - does that output look correct to you? any quick way to test it 
will, if used as an input, procedurally generate the correct python code?


was (Author: kdoran):
[~mcgilman] I added this annotation to 
{{ProcessGroupResource.uploadTemplate(...)}}:

{code}
@ApiImplicitParams(
value = {
@ApiImplicitParam(
name = "body",
value = "The binary content of the template file being uploaded",
required = true,
dataType = "java.io.File",
paramType = "body")
}
)
{code}

which changes the swagger.json output to:

{noformat}
...
"/process-groups/{id}/templates/upload" : {
  "post" : {
"tags" : [ "process-groups" ],
"summary" : "Uploads a template",
"description" : "",
"operationId" : "uploadTemplate",
"consumes" : [ "multipart/form-data" ],
"produces" : [ "application/xml" ],
"parameters" : [ {
  "name" : "id",
  "in" : "path",
  "description" : "The process group id.",
  "required" : true,
  "type" : "string"
}, {
  "in" : "body",
  "name" : "body",
  "description" : "The binary content of the template file being uploaded",
  "required" : true,
  "schema" : {
"type" : "file"
  }
} ],
"responses" : {
  "200" : {
"description" : "successful operation",
"schema" : {
  "$ref" : "#/definitions/TemplateEntity"
}
  },
  "400" : {
"description" : "NiFi was unable to complete the request because it was 
invalid. The request should not be retried without modification."
  },
  "401" : {
"description" : "Client could not be authenticated."
  },
  "403" : {
"description" : "Client is not authorized to make this request."
  },
  "409" : {
"description" : "The request was valid but NiFi was not in the 
appropriate state to process it. Retrying the same request later may be 
successful."
  }
},
"security" : [ {
  "Write - /process-groups/{uuid}" : [ ]
} ]
  }
},
...
{noformat}

[~chaffelson] - does that output look correct to you? any quick way to test it 
will, if used as an input, procedurally generate the correct python code?

> Rest-api swagger definition produces non-functional template import in Python
> -
>
> Key: NIFI-4801
> URL: https://issues.apache.org/jira/browse/NIFI-4801
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0
> Environment: Python 2.7/3.6
>Reporter: Daniel Chaffelson
>Assignee: Kevin Doran
>Priority: Major
>
> The 

[jira] [Commented] (NIFI-4803) MergeContent can create smaller bins than it should, if Merge Strategy changed from 'Defragment' to 'Bin-Packing Algorithm'

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334854#comment-16334854
 ] 

ASF GitHub Bot commented on NIFI-4803:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2423


> MergeContent can create smaller bins than it should, if Merge Strategy 
> changed from 'Defragment' to 'Bin-Packing Algorithm'
> ---
>
> Key: NIFI-4803
> URL: https://issues.apache.org/jira/browse/NIFI-4803
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.6.0
>
>
> If the Merge Strategy is changed from "Defragment" to "Bin-Packing Algorithm" 
> and then send in FlowFiles that have a fragment.count attribute, we end up 
> creating a bin that is no larger than that fragment.count attribute. However, 
> for Bin-Packing Algorithm we should not be inspecting that attribute.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFI-4803) MergeContent can create smaller bins than it should, if Merge Strategy changed from 'Defragment' to 'Bin-Packing Algorithm'

[Bryan Bende (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Bende updated NIFI-4803:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> MergeContent can create smaller bins than it should, if Merge Strategy 
> changed from 'Defragment' to 'Bin-Packing Algorithm'
> ---
>
> Key: NIFI-4803
> URL: https://issues.apache.org/jira/browse/NIFI-4803
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.6.0
>
>
> If the Merge Strategy is changed from "Defragment" to "Bin-Packing Algorithm" 
> and then send in FlowFiles that have a fragment.count attribute, we end up 
> creating a bin that is no larger than that fragment.count attribute. However, 
> for Bin-Packing Algorithm we should not be inspecting that attribute.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2423: NIFI-4803: Ensure that we set the FileCountAttribut...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2423


---

[jira] [Commented] (NIFI-4803) MergeContent can create smaller bins than it should, if Merge Strategy changed from 'Defragment' to 'Bin-Packing Algorithm'

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334852#comment-16334852
 ] 

ASF subversion and git services commented on NIFI-4803:
---

Commit 6f282c6843066f79fc7d53ebec98b4e4bb39f833 in nifi's branch 
refs/heads/master from [~markap14]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=6f282c6 ]

NIFI-4803: Ensure that we set the FileCountAttribute of BinManager to the 
proper value, regardless of the merge strategy

This closes #2423.

Signed-off-by: Bryan Bende 


> MergeContent can create smaller bins than it should, if Merge Strategy 
> changed from 'Defragment' to 'Bin-Packing Algorithm'
> ---
>
> Key: NIFI-4803
> URL: https://issues.apache.org/jira/browse/NIFI-4803
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.6.0
>
>
> If the Merge Strategy is changed from "Defragment" to "Bin-Packing Algorithm" 
> and then send in FlowFiles that have a fragment.count attribute, we end up 
> creating a bin that is no larger than that fragment.count attribute. However, 
> for Bin-Packing Algorithm we should not be inspecting that attribute.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4803) MergeContent can create smaller bins than it should, if Merge Strategy changed from 'Defragment' to 'Bin-Packing Algorithm'

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334851#comment-16334851
 ] 

ASF GitHub Bot commented on NIFI-4803:
--

Github user bbende commented on the issue:

https://github.com/apache/nifi/pull/2423
  
Looks good, will merge


> MergeContent can create smaller bins than it should, if Merge Strategy 
> changed from 'Defragment' to 'Bin-Packing Algorithm'
> ---
>
> Key: NIFI-4803
> URL: https://issues.apache.org/jira/browse/NIFI-4803
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.6.0
>
>
> If the Merge Strategy is changed from "Defragment" to "Bin-Packing Algorithm" 
> and then send in FlowFiles that have a fragment.count attribute, we end up 
> creating a bin that is no larger than that fragment.count attribute. However, 
> for Bin-Packing Algorithm we should not be inspecting that attribute.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi issue #2423: NIFI-4803: Ensure that we set the FileCountAttribute of Bi...

[bbende]

Github user bbende commented on the issue:

https://github.com/apache/nifi/pull/2423
  
Looks good, will merge


---

[jira] [Commented] (NIFIREG-123) Multiple links referenced in NiFi Registry docs should open in a new window, not rendered within the docs themselves

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFIREG-123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334846#comment-16334846
 ] 

ASF GitHub Bot commented on NIFIREG-123:


GitHub user andrewmlim opened a pull request:

https://github.com/apache/nifi-registry/pull/93

NIFIREG-123 Made external links open in new windows



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/andrewmlim/nifi-registry NIFIREG-123

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-registry/pull/93.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #93


commit 040aed46c7ded20da792850c4969097796a41a61
Author: Andrew Lim 
Date:   2018-01-22T20:26:37Z

NIFIREG-123 Made external lnks open in new windows




> Multiple links referenced in NiFi Registry docs should open in a new window, 
> not rendered within the docs themselves
> 
>
> Key: NIFIREG-123
> URL: https://issues.apache.org/jira/browse/NIFIREG-123
> Project: NiFi Registry
>  Issue Type: Improvement
>Affects Versions: 0.1.0
>Reporter: Andrew Lim
>Assignee: Andrew Lim
>Priority: Trivial
>
> Apply changes made in NIFI-4791 for NiFi docs to Registry docs



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi-registry pull request #93: NIFIREG-123 Made external links open in new ...

[andrewmlim]

GitHub user andrewmlim opened a pull request:

https://github.com/apache/nifi-registry/pull/93

NIFIREG-123 Made external links open in new windows



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/andrewmlim/nifi-registry NIFIREG-123

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-registry/pull/93.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #93


commit 040aed46c7ded20da792850c4969097796a41a61
Author: Andrew Lim 
Date:   2018-01-22T20:26:37Z

NIFIREG-123 Made external lnks open in new windows




---

[jira] [Updated] (NIFI-4803) MergeContent can create smaller bins than it should, if Merge Strategy changed from 'Defragment' to 'Bin-Packing Algorithm'

[Mark Payne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Payne updated NIFI-4803:
-
Description: If the Merge Strategy is changed from "Defragment" to 
"Bin-Packing Algorithm" and then send in FlowFiles that have a fragment.count 
attribute, we end up creating a bin that is no larger than that fragment.count 
attribute. However, for Bin-Packing Algorithm we should not be inspecting that 
attribute.

> MergeContent can create smaller bins than it should, if Merge Strategy 
> changed from 'Defragment' to 'Bin-Packing Algorithm'
> ---
>
> Key: NIFI-4803
> URL: https://issues.apache.org/jira/browse/NIFI-4803
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.6.0
>
>
> If the Merge Strategy is changed from "Defragment" to "Bin-Packing Algorithm" 
> and then send in FlowFiles that have a fragment.count attribute, we end up 
> creating a bin that is no larger than that fragment.count attribute. However, 
> for Bin-Packing Algorithm we should not be inspecting that attribute.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4424) org.apache.nifi.NiFi does not allow programmatic access to the NiFi engine

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334812#comment-16334812
 ] 

ASF GitHub Bot commented on NIFI-4424:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2251


> org.apache.nifi.NiFi does not allow programmatic access to the NiFi engine
> --
>
> Key: NIFI-4424
> URL: https://issues.apache.org/jira/browse/NIFI-4424
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Affects Versions: 1.3.0
>Reporter: Peter Horvath
>Priority: Major
> Fix For: 1.6.0
>
>
> Class {{org.apache.nifi.NiFi}} was not designed with extensibility or 
> programmatic access in mind.
> This class is the entry point of the engine, however, the current 
> implementation does not allow
> a potential caller (e.g. an integration test harness) to bootstrap the engine 
> and then shut it down properly:
> The main method {{org.apache.nifi.NiFi#main}} simply logs any exception, 
> which is fine
> when started from the command line, however prevents programmatic usage and
> detecting error conditions (Exceptions) that would be essential to 
> programatically access 
> it from an integration test.
> The constructor {{org.apache.nifi.NiFi#NiFi}} registers an 
> {{UncaughtExceptionHandler}}, 
> a JVM {{Shutdown Hook}} and changes logging framework settings.
> *Please change this behaviour:*
> Expose *two* methods, one of which accepts the command line argument one 
> would pass
> to the NiFi process and another one, which allows the NiFiProperties object 
> to be passed.
> This method should return the {{NiFi}} object instance for further 
> programmatic access.
> The logic used to register {{UncaughtExceptionHandler}}, a JVM Shutdown Hook 
> and 
> changing logging framework settings should be extracted to a {{protected}} 
> *instance*
> method so that a client can override their behaviour with a NO-OP.
> A second class called e.g. {{org.apache.nifi.EmbeddedNiFi}} could be 
> introduced as
> a base class for this use-case, where the engine is started through the Java 
> API.
> *Please note these changes are baby-steps towards the implementation of a 
> NiFi integration test harness.*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (NIFI-4424) org.apache.nifi.NiFi does not allow programmatic access to the NiFi engine

[Andy LoPresto (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andy LoPresto resolved NIFI-4424.
-
   Resolution: Fixed
Fix Version/s: 1.6.0

> org.apache.nifi.NiFi does not allow programmatic access to the NiFi engine
> --
>
> Key: NIFI-4424
> URL: https://issues.apache.org/jira/browse/NIFI-4424
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Affects Versions: 1.3.0
>Reporter: Peter Horvath
>Priority: Major
> Fix For: 1.6.0
>
>
> Class {{org.apache.nifi.NiFi}} was not designed with extensibility or 
> programmatic access in mind.
> This class is the entry point of the engine, however, the current 
> implementation does not allow
> a potential caller (e.g. an integration test harness) to bootstrap the engine 
> and then shut it down properly:
> The main method {{org.apache.nifi.NiFi#main}} simply logs any exception, 
> which is fine
> when started from the command line, however prevents programmatic usage and
> detecting error conditions (Exceptions) that would be essential to 
> programatically access 
> it from an integration test.
> The constructor {{org.apache.nifi.NiFi#NiFi}} registers an 
> {{UncaughtExceptionHandler}}, 
> a JVM {{Shutdown Hook}} and changes logging framework settings.
> *Please change this behaviour:*
> Expose *two* methods, one of which accepts the command line argument one 
> would pass
> to the NiFi process and another one, which allows the NiFiProperties object 
> to be passed.
> This method should return the {{NiFi}} object instance for further 
> programmatic access.
> The logic used to register {{UncaughtExceptionHandler}}, a JVM Shutdown Hook 
> and 
> changing logging framework settings should be extracted to a {{protected}} 
> *instance*
> method so that a client can override their behaviour with a NO-OP.
> A second class called e.g. {{org.apache.nifi.EmbeddedNiFi}} could be 
> introduced as
> a base class for this use-case, where the engine is started through the Java 
> API.
> *Please note these changes are baby-steps towards the implementation of a 
> NiFi integration test harness.*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4424) org.apache.nifi.NiFi does not allow programmatic access to the NiFi engine

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334813#comment-16334813
 ] 

ASF GitHub Bot commented on NIFI-4424:
--

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2251
  
Ran `contrib-check` and all tests pass. Fired up an instance of the 
application and it performed normally. +1, merging. 


> org.apache.nifi.NiFi does not allow programmatic access to the NiFi engine
> --
>
> Key: NIFI-4424
> URL: https://issues.apache.org/jira/browse/NIFI-4424
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Affects Versions: 1.3.0
>Reporter: Peter Horvath
>Priority: Major
> Fix For: 1.6.0
>
>
> Class {{org.apache.nifi.NiFi}} was not designed with extensibility or 
> programmatic access in mind.
> This class is the entry point of the engine, however, the current 
> implementation does not allow
> a potential caller (e.g. an integration test harness) to bootstrap the engine 
> and then shut it down properly:
> The main method {{org.apache.nifi.NiFi#main}} simply logs any exception, 
> which is fine
> when started from the command line, however prevents programmatic usage and
> detecting error conditions (Exceptions) that would be essential to 
> programatically access 
> it from an integration test.
> The constructor {{org.apache.nifi.NiFi#NiFi}} registers an 
> {{UncaughtExceptionHandler}}, 
> a JVM {{Shutdown Hook}} and changes logging framework settings.
> *Please change this behaviour:*
> Expose *two* methods, one of which accepts the command line argument one 
> would pass
> to the NiFi process and another one, which allows the NiFiProperties object 
> to be passed.
> This method should return the {{NiFi}} object instance for further 
> programmatic access.
> The logic used to register {{UncaughtExceptionHandler}}, a JVM Shutdown Hook 
> and 
> changing logging framework settings should be extracted to a {{protected}} 
> *instance*
> method so that a client can override their behaviour with a NO-OP.
> A second class called e.g. {{org.apache.nifi.EmbeddedNiFi}} could be 
> introduced as
> a base class for this use-case, where the engine is started through the Java 
> API.
> *Please note these changes are baby-steps towards the implementation of a 
> NiFi integration test harness.*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi issue #2251: NIFI-4424 org.apache.nifi.NiFi does not allow programmatic...

[alopresto]

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2251
  
Ran `contrib-check` and all tests pass. Fired up an instance of the 
application and it performed normally. +1, merging. 


---

[jira] [Commented] (NIFI-4424) org.apache.nifi.NiFi does not allow programmatic access to the NiFi engine

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334810#comment-16334810
 ] 

ASF subversion and git services commented on NIFI-4424:
---

Commit 3ebfcd5ae5132e29482d571e767142c806e30469 in nifi's branch 
refs/heads/master from [~peter.gergely.horv...@gmail.com]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=3ebfcd5 ]

NIFI-4424 Added functionality to allow NiFi to run in "embedded" mode for 
eventual integration test access.

This closes #2251.

Signed-off-by: Andy LoPresto 


> org.apache.nifi.NiFi does not allow programmatic access to the NiFi engine
> --
>
> Key: NIFI-4424
> URL: https://issues.apache.org/jira/browse/NIFI-4424
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Affects Versions: 1.3.0
>Reporter: Peter Horvath
>Priority: Major
>
> Class {{org.apache.nifi.NiFi}} was not designed with extensibility or 
> programmatic access in mind.
> This class is the entry point of the engine, however, the current 
> implementation does not allow
> a potential caller (e.g. an integration test harness) to bootstrap the engine 
> and then shut it down properly:
> The main method {{org.apache.nifi.NiFi#main}} simply logs any exception, 
> which is fine
> when started from the command line, however prevents programmatic usage and
> detecting error conditions (Exceptions) that would be essential to 
> programatically access 
> it from an integration test.
> The constructor {{org.apache.nifi.NiFi#NiFi}} registers an 
> {{UncaughtExceptionHandler}}, 
> a JVM {{Shutdown Hook}} and changes logging framework settings.
> *Please change this behaviour:*
> Expose *two* methods, one of which accepts the command line argument one 
> would pass
> to the NiFi process and another one, which allows the NiFiProperties object 
> to be passed.
> This method should return the {{NiFi}} object instance for further 
> programmatic access.
> The logic used to register {{UncaughtExceptionHandler}}, a JVM Shutdown Hook 
> and 
> changing logging framework settings should be extracted to a {{protected}} 
> *instance*
> method so that a client can override their behaviour with a NO-OP.
> A second class called e.g. {{org.apache.nifi.EmbeddedNiFi}} could be 
> introduced as
> a base class for this use-case, where the engine is started through the Java 
> API.
> *Please note these changes are baby-steps towards the implementation of a 
> NiFi integration test harness.*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2251: NIFI-4424 org.apache.nifi.NiFi does not allow progr...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2251


---

[jira] [Commented] (NIFI-4803) MergeContent can create smaller bins than it should, if Merge Strategy changed from 'Defragment' to 'Bin-Packing Algorithm'

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334800#comment-16334800
 ] 

ASF GitHub Bot commented on NIFI-4803:
--

GitHub user markap14 opened a pull request:

https://github.com/apache/nifi/pull/2423

NIFI-4803: Ensure that we set the FileCountAttribute of BinManager to…

… the proper value, regardless of the merge strategy

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/markap14/nifi NIFI-4803

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2423.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2423


commit ed4e56d8c1fdda1bdbd85c176e499faf31002ccf
Author: Mark Payne 
Date:   2018-01-22T19:50:19Z

NIFI-4803: Ensure that we set the FileCountAttribute of BinManager to the 
proper value, regardless of the merge strategy




> MergeContent can create smaller bins than it should, if Merge Strategy 
> changed from 'Defragment' to 'Bin-Packing Algorithm'
> ---
>
> Key: NIFI-4803
> URL: https://issues.apache.org/jira/browse/NIFI-4803
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.6.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFI-4803) MergeContent can create smaller bins than it should, if Merge Strategy changed from 'Defragment' to 'Bin-Packing Algorithm'

[Mark Payne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Payne updated NIFI-4803:
-
Status: Patch Available  (was: Open)

> MergeContent can create smaller bins than it should, if Merge Strategy 
> changed from 'Defragment' to 'Bin-Packing Algorithm'
> ---
>
> Key: NIFI-4803
> URL: https://issues.apache.org/jira/browse/NIFI-4803
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.6.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2423: NIFI-4803: Ensure that we set the FileCountAttribut...

[markap14]

GitHub user markap14 opened a pull request:

https://github.com/apache/nifi/pull/2423

NIFI-4803: Ensure that we set the FileCountAttribute of BinManager to…

… the proper value, regardless of the merge strategy

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/markap14/nifi NIFI-4803

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2423.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2423


commit ed4e56d8c1fdda1bdbd85c176e499faf31002ccf
Author: Mark Payne 
Date:   2018-01-22T19:50:19Z

NIFI-4803: Ensure that we set the FileCountAttribute of BinManager to the 
proper value, regardless of the merge strategy




---

[jira] [Created] (NIFI-4803) MergeContent can create smaller bins than it should, if Merge Strategy changed from 'Defragment' to 'Bin-Packing Algorithm'

[Mark Payne (JIRA)]

Mark Payne created NIFI-4803:


 Summary: MergeContent can create smaller bins than it should, if 
Merge Strategy changed from 'Defragment' to 'Bin-Packing Algorithm'
 Key: NIFI-4803
 URL: https://issues.apache.org/jira/browse/NIFI-4803
 Project: Apache NiFi
  Issue Type: Bug
  Components: Extensions
Reporter: Mark Payne
Assignee: Mark Payne
 Fix For: 1.6.0






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFI-4797) Allow NiFi UI to be displayed in a frame

[Andy LoPresto (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andy LoPresto updated NIFI-4797:

Labels: frame security  (was: )

> Allow NiFi UI to be displayed in a frame
> 
>
> Key: NIFI-4797
> URL: https://issues.apache.org/jira/browse/NIFI-4797
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Affects Versions: 1.4.0
>Reporter: Virgil Fritz Taneza
>Priority: Minor
>  Labels: frame, security
>
> We have a web-based solution that is to be served by instances deploy-able in 
> Azure, AWS and VMWare. In this solution, we are trying to incorporate NiFi by 
> starting the service in the same server instance and displaying the NiFi UI 
> in a frame of our said solution. In this setup, we started using NiFi 1.1.2 
> and it worked. Now when we tried to update to 1.4.0, the NiFi UI can't be 
> displayed in a frame due to some error.  I submitted a question to the 
> developer list and one of the suggested workaround was to change the value 
> provided to the response header sent by the Jetty server. I wonder if we can 
> have a configurable solution to this so we don't need to modify the code 
> every time there's new release. 
>  
> Here is the link to the question I submitted to the developer list: 
> http://apache-nifi-developer-list.39713.n7.nabble.com/NiFI-1-4-0-UI-can-t-be-displayed-in-an-IFrame-td17427.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFI-4773) Database Fetch processor setup is incorrect

[Matt Burgess (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Burgess updated NIFI-4773:
---
Affects Version/s: (was: 1.2.0)
   Status: Patch Available  (was: In Progress)

> Database Fetch processor setup is incorrect
> ---
>
> Key: NIFI-4773
> URL: https://issues.apache.org/jira/browse/NIFI-4773
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Wynner
>Assignee: Matt Burgess
>Priority: Major
>
> The QueryDatabaseTable processor attempts to make a database connection 
> during setup (OnScheduled), this can cause issues with the flow when errors 
> occur.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2422: NIFI-4773: Moved DB Fetch processors' connection co...

[mattyb149]

GitHub user mattyb149 opened a pull request:

https://github.com/apache/nifi/pull/2422

NIFI-4773: Moved DB Fetch processors' connection code from setup to 
onTrigger

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [x] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [x] Is your initial contribution a single, squashed commit?

### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mattyb149/nifi NIFI-4773

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2422.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2422


commit ecb09f33b3c23f346a7cc069036088829bdc09ee
Author: Matthew Burgess 
Date:   2018-01-22T18:42:35Z

NIFI-4773: Moved DB Fetch processors' connection code from setup to 
onTrigger




---

[jira] [Updated] (NIFI-4773) Database Fetch processor setup is incorrect

[Matt Burgess (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Burgess updated NIFI-4773:
---
Summary: Database Fetch processor setup is incorrect  (was: 
QueryDatabaseTable setup is incorrect)

> Database Fetch processor setup is incorrect
> ---
>
> Key: NIFI-4773
> URL: https://issues.apache.org/jira/browse/NIFI-4773
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.2.0
>Reporter: Wynner
>Assignee: Matt Burgess
>Priority: Major
>
> The QueryDatabaseTable processor attempts to make a database connection 
> during setup (OnScheduled), this can cause issues with the flow when errors 
> occur.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4773) Database Fetch processor setup is incorrect

[Matt Burgess (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334725#comment-16334725
 ] 

Matt Burgess commented on NIFI-4773:


Under some conditions, GenerateTableFetch will also try to connect to the DB 
from OnScheduled, will need to change that one as well.

> Database Fetch processor setup is incorrect
> ---
>
> Key: NIFI-4773
> URL: https://issues.apache.org/jira/browse/NIFI-4773
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.2.0
>Reporter: Wynner
>Assignee: Matt Burgess
>Priority: Major
>
> The QueryDatabaseTable processor attempts to make a database connection 
> during setup (OnScheduled), this can cause issues with the flow when errors 
> occur.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4424) org.apache.nifi.NiFi does not allow programmatic access to the NiFi engine

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334710#comment-16334710
 ] 

ASF GitHub Bot commented on NIFI-4424:
--

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2251
  
@peter-gergely-horvath I will merge this today. 


> org.apache.nifi.NiFi does not allow programmatic access to the NiFi engine
> --
>
> Key: NIFI-4424
> URL: https://issues.apache.org/jira/browse/NIFI-4424
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Affects Versions: 1.3.0
>Reporter: Peter Horvath
>Priority: Major
>
> Class {{org.apache.nifi.NiFi}} was not designed with extensibility or 
> programmatic access in mind.
> This class is the entry point of the engine, however, the current 
> implementation does not allow
> a potential caller (e.g. an integration test harness) to bootstrap the engine 
> and then shut it down properly:
> The main method {{org.apache.nifi.NiFi#main}} simply logs any exception, 
> which is fine
> when started from the command line, however prevents programmatic usage and
> detecting error conditions (Exceptions) that would be essential to 
> programatically access 
> it from an integration test.
> The constructor {{org.apache.nifi.NiFi#NiFi}} registers an 
> {{UncaughtExceptionHandler}}, 
> a JVM {{Shutdown Hook}} and changes logging framework settings.
> *Please change this behaviour:*
> Expose *two* methods, one of which accepts the command line argument one 
> would pass
> to the NiFi process and another one, which allows the NiFiProperties object 
> to be passed.
> This method should return the {{NiFi}} object instance for further 
> programmatic access.
> The logic used to register {{UncaughtExceptionHandler}}, a JVM Shutdown Hook 
> and 
> changing logging framework settings should be extracted to a {{protected}} 
> *instance*
> method so that a client can override their behaviour with a NO-OP.
> A second class called e.g. {{org.apache.nifi.EmbeddedNiFi}} could be 
> introduced as
> a base class for this use-case, where the engine is started through the Java 
> API.
> *Please note these changes are baby-steps towards the implementation of a 
> NiFi integration test harness.*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi issue #2251: NIFI-4424 org.apache.nifi.NiFi does not allow programmatic...

[alopresto]

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2251
  
@peter-gergely-horvath I will merge this today. 


---

[jira] [Comment Edited] (NIFI-4621) Allow inputs to ListSFTP

[Puspendu Banerjee (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334642#comment-16334642
 ] 

Puspendu Banerjee edited comment on NIFI-4621 at 1/22/18 6:26 PM:
--

Requesting for comment on pros, cons and security implication.

[~soumya.ghosh] Could you please provide a real life usecase for this feature? 
can we go for a limited number of SFTP hosts? Opening this feature for all 
hosts / unlimited unique sftp hosts could become a stepping stone for DDoS 
attack.


was (Author: puspendu.baner...@gmail.com):
Requesting for comment on pros, cons and security implication.

[~soumya.ghosh] Can you provide a real life usecase for this feature? can we go 
for a limited number of SFTP hosts? Opening this feature for all hosts / 
unlimited unique sftp hosts could become a stepping stone for DDoS attack.

> Allow inputs to ListSFTP
> 
>
> Key: NIFI-4621
> URL: https://issues.apache.org/jira/browse/NIFI-4621
> Project: Apache NiFi
>  Issue Type: Improvement
>Affects Versions: 1.4.0
>Reporter: Soumya Shanta Ghosh
>Assignee: Puspendu Banerjee
>Priority: Critical
>
> ListSFTP supports listing of the supplied directory (Remote Path) 
> out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" 
> / "Private Key Passphrase". 
> The password can change at a regular interval (depending on organization 
> policy) or the Hostname or the Remote Path can change based on some other 
> requirement.
> This is a case to allow ListSFTP to leverage the use of Nifi Expression 
> language so that the values of Hostname, Password and/or Remote Path can be 
> set based on the attributes of an incoming flow file.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (NIFI-4621) Allow inputs to ListSFTP

[Puspendu Banerjee (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Puspendu Banerjee reassigned NIFI-4621:
---

Assignee: Puspendu Banerjee

> Allow inputs to ListSFTP
> 
>
> Key: NIFI-4621
> URL: https://issues.apache.org/jira/browse/NIFI-4621
> Project: Apache NiFi
>  Issue Type: Improvement
>Affects Versions: 1.4.0
>Reporter: Soumya Shanta Ghosh
>Assignee: Puspendu Banerjee
>Priority: Critical
>
> ListSFTP supports listing of the supplied directory (Remote Path) 
> out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" 
> / "Private Key Passphrase". 
> The password can change at a regular interval (depending on organization 
> policy) or the Hostname or the Remote Path can change based on some other 
> requirement.
> This is a case to allow ListSFTP to leverage the use of Nifi Expression 
> language so that the values of Hostname, Password and/or Remote Path can be 
> set based on the attributes of an incoming flow file.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP

[Puspendu Banerjee (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334642#comment-16334642
 ] 

Puspendu Banerjee commented on NIFI-4621:
-

Requesting for comment on pros, cons and security implication.

[~soumya.ghosh] Can you provide a real life usecase for this feature? can we go 
for a limited number of SFTP hosts? Opening this feature for all hosts / 
unlimited unique sftp hosts could become a stepping stone for DDoS attack.

> Allow inputs to ListSFTP
> 
>
> Key: NIFI-4621
> URL: https://issues.apache.org/jira/browse/NIFI-4621
> Project: Apache NiFi
>  Issue Type: Improvement
>Affects Versions: 1.4.0
>Reporter: Soumya Shanta Ghosh
>Priority: Critical
>
> ListSFTP supports listing of the supplied directory (Remote Path) 
> out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" 
> / "Private Key Passphrase". 
> The password can change at a regular interval (depending on organization 
> policy) or the Hostname or the Remote Path can change based on some other 
> requirement.
> This is a case to allow ListSFTP to leverage the use of Nifi Expression 
> language so that the values of Hostname, Password and/or Remote Path can be 
> set based on the attributes of an incoming flow file.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFI-4799) UI - Variable Registry - "Close" / referencing processors

[Mark Payne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Payne updated NIFI-4799:
-
   Resolution: Fixed
Fix Version/s: 1.6.0
   Status: Resolved  (was: Patch Available)

> UI - Variable Registry - "Close" / referencing processors
> -
>
> Key: NIFI-4799
> URL: https://issues.apache.org/jira/browse/NIFI-4799
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.5.0
> Environment: Firefox 57.0.4
> NiFi 1.5.0
>Reporter: Pierre Villard
>Assignee: Matt Gilman
>Priority: Major
> Fix For: 1.6.0
>
> Attachments: Screen Shot 2018-01-21 at 12.08.48 AM.png, Screen Shot 
> 2018-01-21 at 12.12.01 AM.png
>
>
> When adding variables or updating in the variable registry of a process 
> group, I see two issues:
>  * after clicking "Apply", operations are executed to apply the changes, but 
> the button "Close" does not show up once operations are completed (first 
> screenshot)
>  * list of referencing processors is not correct: in one of the screenshot it 
> shows a TailFile processor although the processor is not in the process group 
> and does not use expression language



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4799) UI - Variable Registry - "Close" / referencing processors

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334633#comment-16334633
 ] 

ASF GitHub Bot commented on NIFI-4799:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2421


> UI - Variable Registry - "Close" / referencing processors
> -
>
> Key: NIFI-4799
> URL: https://issues.apache.org/jira/browse/NIFI-4799
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.5.0
> Environment: Firefox 57.0.4
> NiFi 1.5.0
>Reporter: Pierre Villard
>Assignee: Matt Gilman
>Priority: Major
> Fix For: 1.6.0
>
> Attachments: Screen Shot 2018-01-21 at 12.08.48 AM.png, Screen Shot 
> 2018-01-21 at 12.12.01 AM.png
>
>
> When adding variables or updating in the variable registry of a process 
> group, I see two issues:
>  * after clicking "Apply", operations are executed to apply the changes, but 
> the button "Close" does not show up once operations are completed (first 
> screenshot)
>  * list of referencing processors is not correct: in one of the screenshot it 
> shows a TailFile processor although the processor is not in the process group 
> and does not use expression language



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4799) UI - Variable Registry - "Close" / referencing processors

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334631#comment-16334631
 ] 

ASF subversion and git services commented on NIFI-4799:
---

Commit 05e9e6eaa5f92a0277ef1a283c6470db08d34de8 in nifi's branch 
refs/heads/master from [~mcgilman]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=05e9e6e ]

NIFI-4799:
- Ensure variable registry update request is marked completed.
This closes #2421.

Signed-off-by: Mark Payne 


> UI - Variable Registry - "Close" / referencing processors
> -
>
> Key: NIFI-4799
> URL: https://issues.apache.org/jira/browse/NIFI-4799
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.5.0
> Environment: Firefox 57.0.4
> NiFi 1.5.0
>Reporter: Pierre Villard
>Assignee: Matt Gilman
>Priority: Major
> Attachments: Screen Shot 2018-01-21 at 12.08.48 AM.png, Screen Shot 
> 2018-01-21 at 12.12.01 AM.png
>
>
> When adding variables or updating in the variable registry of a process 
> group, I see two issues:
>  * after clicking "Apply", operations are executed to apply the changes, but 
> the button "Close" does not show up once operations are completed (first 
> screenshot)
>  * list of referencing processors is not correct: in one of the screenshot it 
> shows a TailFile processor although the processor is not in the process group 
> and does not use expression language



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4799) UI - Variable Registry - "Close" / referencing processors

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334632#comment-16334632
 ] 

ASF GitHub Bot commented on NIFI-4799:
--

Github user markap14 commented on the issue:

https://github.com/apache/nifi/pull/2421
  
@mcgilman was able to verify that this fixes the described behavior. +1 
merged to master. Thanks for knocking that out!


> UI - Variable Registry - "Close" / referencing processors
> -
>
> Key: NIFI-4799
> URL: https://issues.apache.org/jira/browse/NIFI-4799
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.5.0
> Environment: Firefox 57.0.4
> NiFi 1.5.0
>Reporter: Pierre Villard
>Assignee: Matt Gilman
>Priority: Major
> Attachments: Screen Shot 2018-01-21 at 12.08.48 AM.png, Screen Shot 
> 2018-01-21 at 12.12.01 AM.png
>
>
> When adding variables or updating in the variable registry of a process 
> group, I see two issues:
>  * after clicking "Apply", operations are executed to apply the changes, but 
> the button "Close" does not show up once operations are completed (first 
> screenshot)
>  * list of referencing processors is not correct: in one of the screenshot it 
> shows a TailFile processor although the processor is not in the process group 
> and does not use expression language



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2421: NIFI-4799: Ensure variable registry update request ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2421


---

[GitHub] nifi issue #2421: NIFI-4799: Ensure variable registry update request is mark...

[markap14]

Github user markap14 commented on the issue:

https://github.com/apache/nifi/pull/2421
  
@mcgilman was able to verify that this fixes the described behavior. +1 
merged to master. Thanks for knocking that out!


---

[jira] [Commented] (MINIFICPP-378) New threads should detach when battery monitor reduces them

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334618#comment-16334618
 ] 

ASF GitHub Bot commented on MINIFICPP-378:
--

GitHub user phrocker opened a pull request:

https://github.com/apache/nifi-minifi-cpp/pull/247

MINIFICPP-378: Detach new threads if daemon threads is true

Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
 in the commit message?

- [ ] Does your PR title start with MINIFI- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file?
- [ ] If applicable, have you updated the NOTICE file?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFICPP-378

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi-cpp/pull/247.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #247


commit 35edf3c3f9f2c8063340bd7d68b485f49c608e37
Author: Marc Parisi 
Date:   2018-01-22T16:45:54Z

MINIFICPP-378: Detach new threads if daemon threads is true




> New threads should detach when battery monitor reduces them
> ---
>
> Key: MINIFICPP-378
> URL: https://issues.apache.org/jira/browse/MINIFICPP-378
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Affects Versions: 0.4.0
>Reporter: marco polo
>Assignee: marco polo
>Priority: Major
> Fix For: 0.4.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi-minifi-cpp pull request #247: MINIFICPP-378: Detach new threads if daem...

[phrocker]

GitHub user phrocker opened a pull request:

https://github.com/apache/nifi-minifi-cpp/pull/247

MINIFICPP-378: Detach new threads if daemon threads is true

Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
 in the commit message?

- [ ] Does your PR title start with MINIFI- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file?
- [ ] If applicable, have you updated the NOTICE file?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFICPP-378

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi-cpp/pull/247.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #247


commit 35edf3c3f9f2c8063340bd7d68b485f49c608e37
Author: Marc Parisi 
Date:   2018-01-22T16:45:54Z

MINIFICPP-378: Detach new threads if daemon threads is true




---

[jira] [Updated] (NIFI-4801) Rest-api swagger definition produces non-functional template import in Python

[Daniel Chaffelson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Chaffelson updated NIFI-4801:

Priority: Major  (was: Minor)

> Rest-api swagger definition produces non-functional template import in Python
> -
>
> Key: NIFI-4801
> URL: https://issues.apache.org/jira/browse/NIFI-4801
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0
> Environment: Python 2.7/3.6
>Reporter: Daniel Chaffelson
>Assignee: Kevin Doran
>Priority: Major
>
> The swagger.json produced when compiling NiFi-1.5.0 results in a 
> ProcessgroupsApi().upload_template function that only accepts the id of the 
> Process Group to receive the template, and no option to specify the template 
> itself.
> It would appear that the underlying API call expects the template to be the 
> body of the request, but the produced function does not allow it to be 
> specified. This is changed from NiFi-1.2.0 where a  'template' keyword 
> argument was included.
> It may also be related to how the TemplatesApi().export_template function 
> used to produce a TemplateDTO and now produces a string. 
> I am unsure in which version since 1.2.0 this changed, it may not 
> specifically be just 1.5.0 code.
> An example of the procedurally generated code can be found at:
> [https://github.com/Chaffelson/nifi-python-swagger-client/blob/master/swagger_client/apis/processgroups_api.py]
> And documentation at:
> [http://nifi-python-swagger-client.readthedocs.io/en/latest/ProcessgroupsApi/#upload_template]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFI-4801) Rest-api swagger definition produces non-functional template import in Python

[Daniel Chaffelson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Chaffelson updated NIFI-4801:

Affects Version/s: 1.6.0

> Rest-api swagger definition produces non-functional template import in Python
> -
>
> Key: NIFI-4801
> URL: https://issues.apache.org/jira/browse/NIFI-4801
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0
> Environment: Python 2.7/3.6
>Reporter: Daniel Chaffelson
>Assignee: Kevin Doran
>Priority: Major
>
> The swagger.json produced when compiling NiFi-1.5.0 results in a 
> ProcessgroupsApi().upload_template function that only accepts the id of the 
> Process Group to receive the template, and no option to specify the template 
> itself.
> It would appear that the underlying API call expects the template to be the 
> body of the request, but the produced function does not allow it to be 
> specified. This is changed from NiFi-1.2.0 where a  'template' keyword 
> argument was included.
> It may also be related to how the TemplatesApi().export_template function 
> used to produce a TemplateDTO and now produces a string. 
> I am unsure in which version since 1.2.0 this changed, it may not 
> specifically be just 1.5.0 code.
> An example of the procedurally generated code can be found at:
> [https://github.com/Chaffelson/nifi-python-swagger-client/blob/master/swagger_client/apis/processgroups_api.py]
> And documentation at:
> [http://nifi-python-swagger-client.readthedocs.io/en/latest/ProcessgroupsApi/#upload_template]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4801) Rest-api swagger definition produces non-functional template import in Python

[Daniel Chaffelson (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334583#comment-16334583
 ] 

Daniel Chaffelson commented on NIFI-4801:
-

[~mcgilman] Yeah this duplicates that issue, though perhaps with a little more 
detail and test code/documentation to look at. Up to you which to move forward.

I would also suggest that you look at the way Export Template works to make 
sure it's producing a consistent experience perhaps?

> Rest-api swagger definition produces non-functional template import in Python
> -
>
> Key: NIFI-4801
> URL: https://issues.apache.org/jira/browse/NIFI-4801
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.5.0
> Environment: Python 2.7/3.6
>Reporter: Daniel Chaffelson
>Assignee: Kevin Doran
>Priority: Minor
>
> The swagger.json produced when compiling NiFi-1.5.0 results in a 
> ProcessgroupsApi().upload_template function that only accepts the id of the 
> Process Group to receive the template, and no option to specify the template 
> itself.
> It would appear that the underlying API call expects the template to be the 
> body of the request, but the produced function does not allow it to be 
> specified. This is changed from NiFi-1.2.0 where a  'template' keyword 
> argument was included.
> It may also be related to how the TemplatesApi().export_template function 
> used to produce a TemplateDTO and now produces a string. 
> I am unsure in which version since 1.2.0 this changed, it may not 
> specifically be just 1.5.0 code.
> An example of the procedurally generated code can be found at:
> [https://github.com/Chaffelson/nifi-python-swagger-client/blob/master/swagger_client/apis/processgroups_api.py]
> And documentation at:
> [http://nifi-python-swagger-client.readthedocs.io/en/latest/ProcessgroupsApi/#upload_template]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFI-4799) UI - Variable Registry - "Close" / referencing processors

[Matt Gilman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Gilman updated NIFI-4799:
--
Status: Patch Available  (was: In Progress)

> UI - Variable Registry - "Close" / referencing processors
> -
>
> Key: NIFI-4799
> URL: https://issues.apache.org/jira/browse/NIFI-4799
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.5.0
> Environment: Firefox 57.0.4
> NiFi 1.5.0
>Reporter: Pierre Villard
>Assignee: Matt Gilman
>Priority: Major
> Attachments: Screen Shot 2018-01-21 at 12.08.48 AM.png, Screen Shot 
> 2018-01-21 at 12.12.01 AM.png
>
>
> When adding variables or updating in the variable registry of a process 
> group, I see two issues:
>  * after clicking "Apply", operations are executed to apply the changes, but 
> the button "Close" does not show up once operations are completed (first 
> screenshot)
>  * list of referencing processors is not correct: in one of the screenshot it 
> shows a TailFile processor although the processor is not in the process group 
> and does not use expression language



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4799) UI - Variable Registry - "Close" / referencing processors

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334551#comment-16334551
 ] 

ASF GitHub Bot commented on NIFI-4799:
--

GitHub user mcgilman opened a pull request:

https://github.com/apache/nifi/pull/2421

NIFI-4799: Ensure variable registry update request is marked complete

NIFI-4799:
- Ensure variable registry update request is marked completed.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mcgilman/nifi NIFI-4799

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2421.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2421


commit a42e21a9fe290038075d0bd873843219a30f296e
Author: Matt Gilman 
Date:   2018-01-22T17:09:15Z

NIFI-4799:
- Ensure variable registry update request is marked completed.




> UI - Variable Registry - "Close" / referencing processors
> -
>
> Key: NIFI-4799
> URL: https://issues.apache.org/jira/browse/NIFI-4799
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.5.0
> Environment: Firefox 57.0.4
> NiFi 1.5.0
>Reporter: Pierre Villard
>Assignee: Matt Gilman
>Priority: Major
> Attachments: Screen Shot 2018-01-21 at 12.08.48 AM.png, Screen Shot 
> 2018-01-21 at 12.12.01 AM.png
>
>
> When adding variables or updating in the variable registry of a process 
> group, I see two issues:
>  * after clicking "Apply", operations are executed to apply the changes, but 
> the button "Close" does not show up once operations are completed (first 
> screenshot)
>  * list of referencing processors is not correct: in one of the screenshot it 
> shows a TailFile processor although the processor is not in the process group 
> and does not use expression language



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2421: NIFI-4799: Ensure variable registry update request ...

[mcgilman]

GitHub user mcgilman opened a pull request:

https://github.com/apache/nifi/pull/2421

NIFI-4799: Ensure variable registry update request is marked complete

NIFI-4799:
- Ensure variable registry update request is marked completed.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mcgilman/nifi NIFI-4799

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2421.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2421


commit a42e21a9fe290038075d0bd873843219a30f296e
Author: Matt Gilman 
Date:   2018-01-22T17:09:15Z

NIFI-4799:
- Ensure variable registry update request is marked completed.




---

[jira] [Created] (MINIFICPP-378) New threads should detach when battery monitor reduces them

[marco polo (JIRA)]

marco polo created MINIFICPP-378:


 Summary: New threads should detach when battery monitor reduces 
them
 Key: MINIFICPP-378
 URL: https://issues.apache.org/jira/browse/MINIFICPP-378
 Project: NiFi MiNiFi C++
  Issue Type: Improvement
Affects Versions: 0.4.0
Reporter: marco polo
Assignee: marco polo
 Fix For: 0.4.0






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4799) UI - Variable Registry - "Close" / referencing processors

[Pierre Villard (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334517#comment-16334517
 ] 

Pierre Villard commented on NIFI-4799:
--

Today, while checking if this also happens with Chrome, I couldn't reproduce 
the referencing processors issue, I think we can forget about it for now. But 
the "Close" button can be reproduced in any browser.

> UI - Variable Registry - "Close" / referencing processors
> -
>
> Key: NIFI-4799
> URL: https://issues.apache.org/jira/browse/NIFI-4799
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.5.0
> Environment: Firefox 57.0.4
> NiFi 1.5.0
>Reporter: Pierre Villard
>Assignee: Matt Gilman
>Priority: Major
> Attachments: Screen Shot 2018-01-21 at 12.08.48 AM.png, Screen Shot 
> 2018-01-21 at 12.12.01 AM.png
>
>
> When adding variables or updating in the variable registry of a process 
> group, I see two issues:
>  * after clicking "Apply", operations are executed to apply the changes, but 
> the button "Close" does not show up once operations are completed (first 
> screenshot)
>  * list of referencing processors is not correct: in one of the screenshot it 
> shows a TailFile processor although the processor is not in the process group 
> and does not use expression language



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (NIFI-4799) UI - Variable Registry - "Close" / referencing processors

[Matt Gilman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Gilman reassigned NIFI-4799:
-

Assignee: Matt Gilman

> UI - Variable Registry - "Close" / referencing processors
> -
>
> Key: NIFI-4799
> URL: https://issues.apache.org/jira/browse/NIFI-4799
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.5.0
> Environment: Firefox 57.0.4
> NiFi 1.5.0
>Reporter: Pierre Villard
>Assignee: Matt Gilman
>Priority: Major
> Attachments: Screen Shot 2018-01-21 at 12.08.48 AM.png, Screen Shot 
> 2018-01-21 at 12.12.01 AM.png
>
>
> When adding variables or updating in the variable registry of a process 
> group, I see two issues:
>  * after clicking "Apply", operations are executed to apply the changes, but 
> the button "Close" does not show up once operations are completed (first 
> screenshot)
>  * list of referencing processors is not correct: in one of the screenshot it 
> shows a TailFile processor although the processor is not in the process group 
> and does not use expression language



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (MINIFICPP-376) EL fails on chained function calls

[Andrew Christianson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Christianson resolved MINIFICPP-376.
---
Resolution: Fixed

> EL fails on chained function calls
> --
>
> Key: MINIFICPP-376
> URL: https://issues.apache.org/jira/browse/MINIFICPP-376
> Project: NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Andrew Christianson
>Assignee: Andrew Christianson
>Priority: Major
>
> Chained calls such as random():mod(10):plus(1) do not work.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (NIFI-4802) PutSQS not UTF-8 encoding output

[Richard St. John (JIRA)]

Richard St. John created NIFI-4802:
--

 Summary: PutSQS not UTF-8 encoding output
 Key: NIFI-4802
 URL: https://issues.apache.org/jira/browse/NIFI-4802
 Project: Apache NiFi
  Issue Type: Bug
  Components: Extensions
Affects Versions: 1.4.0
 Environment: OSX, Linux
Reporter: Richard St. John


PutSQS processor does not properly encode content.  Line 117 reads, 
{code:java}
final string flowFileContent = baos.toString();
{code}

Instead, the output stream should default to UTF-8, or have a configuration 
similar to GetSQS.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (MINIFICPP-377) Resolve test failures as a result of logging changes.

[marco polo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

marco polo updated MINIFICPP-377:
-
Status: Patch Available  (was: Open)

> Resolve test failures as a result of logging changes. 
> --
>
> Key: MINIFICPP-377
> URL: https://issues.apache.org/jira/browse/MINIFICPP-377
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Affects Versions: 0.4.0
>Reporter: marco polo
>Assignee: marco polo
>Priority: Blocker
> Fix For: 0.4.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4787) create HashExpression Processor

[Sourav Gulati (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334380#comment-16334380
 ] 

Sourav Gulati commented on NIFI-4787:
-

I have started working on it. Please assign it to me

> create HashExpression Processor
> ---
>
> Key: NIFI-4787
> URL: https://issues.apache.org/jira/browse/NIFI-4787
> Project: Apache NiFi
>  Issue Type: Wish
>Reporter: Alois Gruber
>Priority: Minor
>  Labels: processor
>
> in addition to HashAttribute and HashContent, a processor would be desirable 
> that creates a hash based on an NiFi Expression
> the processor is based on HashContent
> additional property:
> Expression: a Nifi Expression, the result of the expression is used as input 
> for the hashing
> Alternativly, the HashContent Processor could be extended with this property 
> and in case the expression property is empty, it works like it is working now



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MINIFICPP-377) Resolve test failures as a result of logging changes.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334371#comment-16334371
 ] 

ASF GitHub Bot commented on MINIFICPP-377:
--

GitHub user phrocker opened a pull request:

https://github.com/apache/nifi-minifi-cpp/pull/246

MINIFICPP-377: Change log level for InvokeHTTP to trace

Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
 in the commit message?

- [ ] Does your PR title start with MINIFI- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file?
- [ ] If applicable, have you updated the NOTICE file?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFICPP-377

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi-cpp/pull/246.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #246


commit 28b8f3099f1bb0ef00a51aa4946a2b6ce2bdbadb
Author: Marc Parisi 
Date:   2018-01-22T15:03:57Z

MINIFICPP-377: Change log level for InvokeHTTP to trace




> Resolve test failures as a result of logging changes. 
> --
>
> Key: MINIFICPP-377
> URL: https://issues.apache.org/jira/browse/MINIFICPP-377
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Affects Versions: 0.4.0
>Reporter: marco polo
>Assignee: marco polo
>Priority: Blocker
> Fix For: 0.4.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi-minifi-cpp pull request #246: MINIFICPP-377: Change log level for Invok...

[phrocker]

GitHub user phrocker opened a pull request:

https://github.com/apache/nifi-minifi-cpp/pull/246

MINIFICPP-377: Change log level for InvokeHTTP to trace

Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
 in the commit message?

- [ ] Does your PR title start with MINIFI- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file?
- [ ] If applicable, have you updated the NOTICE file?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFICPP-377

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi-cpp/pull/246.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #246


commit 28b8f3099f1bb0ef00a51aa4946a2b6ce2bdbadb
Author: Marc Parisi 
Date:   2018-01-22T15:03:57Z

MINIFICPP-377: Change log level for InvokeHTTP to trace




---

[jira] [Created] (MINIFICPP-377) Resolve test failures as a result of logging changes.

[marco polo (JIRA)]

marco polo created MINIFICPP-377:


 Summary: Resolve test failures as a result of logging changes. 
 Key: MINIFICPP-377
 URL: https://issues.apache.org/jira/browse/MINIFICPP-377
 Project: NiFi MiNiFi C++
  Issue Type: Improvement
Affects Versions: 0.4.0
Reporter: marco polo
Assignee: marco polo
 Fix For: 0.4.0






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MINIFICPP-376) EL fails on chained function calls

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334361#comment-16334361
 ] 

ASF GitHub Bot commented on MINIFICPP-376:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi-cpp/pull/245


> EL fails on chained function calls
> --
>
> Key: MINIFICPP-376
> URL: https://issues.apache.org/jira/browse/MINIFICPP-376
> Project: NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Andrew Christianson
>Assignee: Andrew Christianson
>Priority: Major
>
> Chained calls such as random():mod(10):plus(1) do not work.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi-minifi-cpp pull request #245: MINIFICPP-376 Fixed failure to apply chai...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi-cpp/pull/245


---

[jira] [Resolved] (NIFI-4744) Detect incorrect configuration when using composite user group provider

[Bryan Bende (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Bende resolved NIFI-4744.
---
   Resolution: Fixed
Fix Version/s: 1.6.0

> Detect incorrect configuration when using composite user group provider
> ---
>
> Key: NIFI-4744
> URL: https://issues.apache.org/jira/browse/NIFI-4744
> Project: Apache NiFi
>  Issue Type: Improvement
>Affects Versions: 1.5.0
>Reporter: Bryan Bende
>Assignee: Kevin Doran
>Priority: Minor
> Fix For: 1.6.0
>
>
> We should be able to detect if the same user group provider is entered more 
> than once in the composite user group provider and fail start up.
> Example config that should fail start up:
> 
> composite-configurable-user-group-provider
> 
> org.apache.nifi.authorization.CompositeConfigurableUserGroupProvider
> file-user-group-provider
> file-user-group-provider
> ldap-user-group-provider
> 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4744) Detect incorrect configuration when using composite user group provider

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334348#comment-16334348
 ] 

ASF GitHub Bot commented on NIFI-4744:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2419


> Detect incorrect configuration when using composite user group provider
> ---
>
> Key: NIFI-4744
> URL: https://issues.apache.org/jira/browse/NIFI-4744
> Project: Apache NiFi
>  Issue Type: Improvement
>Affects Versions: 1.5.0
>Reporter: Bryan Bende
>Assignee: Kevin Doran
>Priority: Minor
> Fix For: 1.6.0
>
>
> We should be able to detect if the same user group provider is entered more 
> than once in the composite user group provider and fail start up.
> Example config that should fail start up:
> 
> composite-configurable-user-group-provider
> 
> org.apache.nifi.authorization.CompositeConfigurableUserGroupProvider
> file-user-group-provider
> file-user-group-provider
> ldap-user-group-provider
> 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2419: NIFI-4744 Detect incorrect authorizers config

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2419


---

[jira] [Commented] (NIFI-4744) Detect incorrect configuration when using composite user group provider

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334347#comment-16334347
 ] 

ASF subversion and git services commented on NIFI-4744:
---

Commit 5d5edfa8960bf7785200912959eb69f313c3c181 in nifi's branch 
refs/heads/master from [~kdoran]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=5d5edfa ]

NIFI-4744 Detect incorrect authorizers config

Adds stricter checks in AuthorizerFactoryBean for unique ids within
a given type of provider and requires unique providers in composite
and composite-configurable user group providers. Failed validation
checks cause startup to fail. Adds test cases for these new rules.

This closes #2419.

Signed-off-by: Bryan Bende 


> Detect incorrect configuration when using composite user group provider
> ---
>
> Key: NIFI-4744
> URL: https://issues.apache.org/jira/browse/NIFI-4744
> Project: Apache NiFi
>  Issue Type: Improvement
>Affects Versions: 1.5.0
>Reporter: Bryan Bende
>Assignee: Kevin Doran
>Priority: Minor
>
> We should be able to detect if the same user group provider is entered more 
> than once in the composite user group provider and fail start up.
> Example config that should fail start up:
> 
> composite-configurable-user-group-provider
> 
> org.apache.nifi.authorization.CompositeConfigurableUserGroupProvider
> file-user-group-provider
> file-user-group-provider
> ldap-user-group-provider
> 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4744) Detect incorrect configuration when using composite user group provider

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334346#comment-16334346
 ] 

ASF GitHub Bot commented on NIFI-4744:
--

Github user bbende commented on the issue:

https://github.com/apache/nifi/pull/2419
  
Looks good, mergin


> Detect incorrect configuration when using composite user group provider
> ---
>
> Key: NIFI-4744
> URL: https://issues.apache.org/jira/browse/NIFI-4744
> Project: Apache NiFi
>  Issue Type: Improvement
>Affects Versions: 1.5.0
>Reporter: Bryan Bende
>Assignee: Kevin Doran
>Priority: Minor
>
> We should be able to detect if the same user group provider is entered more 
> than once in the composite user group provider and fail start up.
> Example config that should fail start up:
> 
> composite-configurable-user-group-provider
> 
> org.apache.nifi.authorization.CompositeConfigurableUserGroupProvider
> file-user-group-provider
> file-user-group-provider
> ldap-user-group-provider
> 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi issue #2419: NIFI-4744 Detect incorrect authorizers config

[bbende]

Github user bbende commented on the issue:

https://github.com/apache/nifi/pull/2419
  
Looks good, mergin


---

[GitHub] nifi issue #2180: Added GetMongoAggregation to support running Mongo aggrega...

[MikeThomsen]

Github user MikeThomsen commented on the issue:

https://github.com/apache/nifi/pull/2180
  
@mattyb149 @alopresto I think all of the changes are in now. Any chance I 
could get this merged?


---

[jira] [Commented] (NIFI-4801) Rest-api swagger definition produces non-functional template import in Python

[Matt Gilman (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334319#comment-16334319
 ] 

Matt Gilman commented on NIFI-4801:
---

[~chaffelson] Is this the same issue as 
https://issues.apache.org/jira/browse/NIFI-4796

> Rest-api swagger definition produces non-functional template import in Python
> -
>
> Key: NIFI-4801
> URL: https://issues.apache.org/jira/browse/NIFI-4801
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.5.0
> Environment: Python 2.7/3.6
>Reporter: Daniel Chaffelson
>Priority: Minor
>
> The swagger.json produced when compiling NiFi-1.5.0 results in a 
> ProcessgroupsApi().upload_template function that only accepts the id of the 
> Process Group to receive the template, and no option to specify the template 
> itself.
> It would appear that the underlying API call expects the template to be the 
> body of the request, but the produced function does not allow it to be 
> specified. This is changed from NiFi-1.2.0 where a  'template' keyword 
> argument was included.
> It may also be related to how the TemplatesApi().export_template function 
> used to produce a TemplateDTO and now produces a string. 
> I am unsure in which version since 1.2.0 this changed, it may not 
> specifically be just 1.5.0 code.
> An example of the procedurally generated code can be found at:
> [https://github.com/Chaffelson/nifi-python-swagger-client/blob/master/swagger_client/apis/processgroups_api.py]
> And documentation at:
> [http://nifi-python-swagger-client.readthedocs.io/en/latest/ProcessgroupsApi/#upload_template]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4731) BigQuery processors

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334280#comment-16334280
 ] 

ASF GitHub Bot commented on NIFI-4731:
--

Github user MikeThomsen commented on the issue:

https://github.com/apache/nifi/pull/2420
  
@nologic Log said you have 164 checkstyle violations in nifi-gcp-processors.


> BigQuery processors
> ---
>
> Key: NIFI-4731
> URL: https://issues.apache.org/jira/browse/NIFI-4731
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Reporter: Mikhail Sosonkin
>Priority: Major
>
> NIFI should have processors for putting data into BigQuery (Streaming and 
> Batch).
> Initial working processors can be found this repository: 
> https://github.com/nologic/nifi/tree/NIFI-4731/nifi-nar-bundles/nifi-gcp-bundle/nifi-gcp-processors/src/main/java/org/apache/nifi/processors/gcp/bigquery
> I'd like to get them into Nifi proper.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi issue #2420: NIFI-4731

[MikeThomsen]

Github user MikeThomsen commented on the issue:

https://github.com/apache/nifi/pull/2420
  
@nologic Log said you have 164 checkstyle violations in nifi-gcp-processors.


---

[jira] [Commented] (NIFI-4325) Create a new ElasticSearch processor that supports the JSON DSL

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334270#comment-16334270
 ] 

ASF GitHub Bot commented on NIFI-4325:
--

Github user MikeThomsen commented on the issue:

https://github.com/apache/nifi/pull/2113
  
@mattyb149 I moved it to a new NAR and rebased against the latest master 
last night. So once this is committed, I'll start working on the new processors 
and service methods.


> Create a new ElasticSearch processor that supports the JSON DSL
> ---
>
> Key: NIFI-4325
> URL: https://issues.apache.org/jira/browse/NIFI-4325
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Mike Thomsen
>Priority: Minor
>
> The existing ElasticSearch processors use the Lucene-style syntax for 
> querying, not the JSON DSL. A new processor is needed that can take a full 
> JSON query and execute it. It should also support aggregation queries in this 
> syntax. A user needs to be able to take a query as-is from Kibana and drop it 
> into NiFi and have it just run.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi issue #2113: NIFI-4325 Added new processor that uses the JSON DSL.

[MikeThomsen]

Github user MikeThomsen commented on the issue:

https://github.com/apache/nifi/pull/2113
  
@mattyb149 I moved it to a new NAR and rebased against the latest master 
last night. So once this is committed, I'll start working on the new processors 
and service methods.


---

[jira] [Updated] (NIFI-4801) Rest-api swagger definition produces non-functional template import in Python

[Daniel Chaffelson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Chaffelson updated NIFI-4801:

Affects Version/s: (was: 1.2.0)

> Rest-api swagger definition produces non-functional template import in Python
> -
>
> Key: NIFI-4801
> URL: https://issues.apache.org/jira/browse/NIFI-4801
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.5.0
> Environment: Python 2.7/3.6
>Reporter: Daniel Chaffelson
>Priority: Minor
>
> The swagger.json produced when compiling NiFi-1.5.0 results in a 
> ProcessgroupsApi().upload_template function that only accepts the id of the 
> Process Group to receive the template, and no option to specify the template 
> itself.
> It would appear that the underlying API call expects the template to be the 
> body of the request, but the produced function does not allow it to be 
> specified. This is changed from NiFi-1.2.0 where a  'template' keyword 
> argument was included.
> It may also be related to how the TemplatesApi().export_template function 
> used to produce a TemplateDTO and now produces a string. 
> I am unsure in which version since 1.2.0 this changed, it may not 
> specifically be just 1.5.0 code.
> An example of the procedurally generated code can be found at:
> [https://github.com/Chaffelson/nifi-python-swagger-client/blob/master/swagger_client/apis/processgroups_api.py]
> And documentation at:
> [http://nifi-python-swagger-client.readthedocs.io/en/latest/ProcessgroupsApi/#upload_template]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-1706) Extend QueryDatabaseTable to support arbitrary queries

[Koji Kawamura (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334006#comment-16334006
 ] 

Koji Kawamura commented on NIFI-1706:
-

[~pee...@gmail.com] You can define select queries in the target RDBMS as VIEW. 
This is an example of MySQL, you can use JOIN in VIEW to provide rich data. 
https://stackoverflow.com/questions/12352048/mysql-create-view-joining-two-tables

> Extend QueryDatabaseTable to support arbitrary queries
> --
>
> Key: NIFI-1706
> URL: https://issues.apache.org/jira/browse/NIFI-1706
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Affects Versions: 1.4.0
>Reporter: Paul Bormans
>Assignee: Peter Wicks
>Priority: Major
>  Labels: features
>
> The QueryDatabaseTable is able to observe a configured database table for new 
> rows and yield these into the flowfile. The model of an rdbms however is 
> often (if not always) normalized so you would need to join various tables in 
> order to "flatten" the data into useful events for a processing pipeline as 
> can be build with nifi or various tools within the hadoop ecosystem.
> The request is to extend the processor to specify an arbitrary sql query 
> instead of specifying the table name + columns.
> In addition (this may be another issue?) it is desired to limit the number of 
> rows returned per run. Not just because of bandwidth issue's from the nifi 
> pipeline onwards but mainly because huge databases may not be able to return 
> so many records within a reasonable time.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-1706) Extend QueryDatabaseTable to support arbitrary queries

[Paul Bormans (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333992#comment-16333992
 ] 

Paul Bormans commented on NIFI-1706:


Hello Koji,

I'm not familiar with the Views concept (have been "out" for some some since i 
wrote the ticket), but i still feel that the processor is not very useful if 
the aim is to extract "rich" data from an rdbms without support for a more 
complex query. The only alternative that you then have is to extract on 
per-table basis and join somewhere else...

Paul

 

> Extend QueryDatabaseTable to support arbitrary queries
> --
>
> Key: NIFI-1706
> URL: https://issues.apache.org/jira/browse/NIFI-1706
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Affects Versions: 1.4.0
>Reporter: Paul Bormans
>Assignee: Peter Wicks
>Priority: Major
>  Labels: features
>
> The QueryDatabaseTable is able to observe a configured database table for new 
> rows and yield these into the flowfile. The model of an rdbms however is 
> often (if not always) normalized so you would need to join various tables in 
> order to "flatten" the data into useful events for a processing pipeline as 
> can be build with nifi or various tools within the hadoop ecosystem.
> The request is to extend the processor to specify an arbitrary sql query 
> instead of specifying the table name + columns.
> In addition (this may be another issue?) it is desired to limit the number of 
> rows returned per run. Not just because of bandwidth issue's from the nifi 
> pipeline onwards but mainly because huge databases may not be able to return 
> so many records within a reasonable time.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)