[jira] [Commented] (NIFI-4256) Add support for all AWS S3 Encryption Options

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121066#comment-16121066
 ] 

ASF GitHub Bot commented on NIFI-4256:
--

Github user jvwing commented on the issue:

https://github.com/apache/nifi/pull/2066
  
@baank Thanks for putting together this PR, it looks like you put a lot of 
thought into covering all the possible encryption scenarios.  I haven't run it 
yet, but I have a few starter questions after looking over some of the code:

1. What was the driver behind updating the AWS SDK version?
1. Although the service interfaces and their methods are named specific to 
encryption, the substance of their interaction are not necessarily limited to 
encryption.  What would you think about making the interfaces more generic?  
For example:
* Could the S3ClientSideEncryptionService be "S3ClientService" with 
only `getClient` methods, with the `needsEncryptedClient()` logic being 
performed internally by the concrete implementation 
StandardS3ClientSideEncryptionService.  I can see a number of use cases beyond 
encryption that could be covered by a custom client factory.
* Could the S3ServerSideEncryptionService be a more generic S3 put 
request modifier?  My efforts at thinking up a good name failed miserably here. 
 But the interface allows many non-encryption modifications of an S3 request, 
which might indeed be useful, despite the `encrypt()` naming of the methods.


> Add support for all AWS S3 Encryption Options
> -
>
> Key: NIFI-4256
> URL: https://issues.apache.org/jira/browse/NIFI-4256
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Affects Versions: 1.2.0
>Reporter: Franco
>  Labels: aws, aws-s3, security
> Fix For: 1.4.0
>
>
> NiFi currently only supports SSE-S3 encryption (AES256).
> Support needs to be added for:
> * SSE-S3
> * SSE-KMS
> * SSE-C
> * CSE-KMS CMK
> * CSE-Master Key
> With all of the appropriate configuration options and such that SSE is 
> available only for PutS3Object whilst CSE is available also for FetchS3Object.
> Given that this will add another 20 or so UI properties the intention is to 
> split it into a Client Side Encryption Service and Server Side Encryption 
> Service. This will allow users to reuse "encryption" across different 
> workflows.
> Existing flows using the Server Side Encryption option will still work as is 
> but will be overridden if a service is added.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi issue #2066: NIFI-4256 - Add support for all AWS S3 Encryption Options

[jvwing]

Github user jvwing commented on the issue:

https://github.com/apache/nifi/pull/2066
  
@baank Thanks for putting together this PR, it looks like you put a lot of 
thought into covering all the possible encryption scenarios.  I haven't run it 
yet, but I have a few starter questions after looking over some of the code:

1. What was the driver behind updating the AWS SDK version?
1. Although the service interfaces and their methods are named specific to 
encryption, the substance of their interaction are not necessarily limited to 
encryption.  What would you think about making the interfaces more generic?  
For example:
* Could the S3ClientSideEncryptionService be "S3ClientService" with 
only `getClient` methods, with the `needsEncryptedClient()` logic being 
performed internally by the concrete implementation 
StandardS3ClientSideEncryptionService.  I can see a number of use cases beyond 
encryption that could be covered by a custom client factory.
* Could the S3ServerSideEncryptionService be a more generic S3 put 
request modifier?  My efforts at thinking up a good name failed miserably here. 
 But the interface allows many non-encryption modifications of an S3 request, 
which might indeed be useful, despite the `encrypt()` naming of the methods.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-4278) ValidateXml - add validation error as an attribute of invalid flow files

[Andy LoPresto (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andy LoPresto updated NIFI-4278:

   Resolution: Fixed
Fix Version/s: 1.4.0
   Status: Resolved  (was: Patch Available)

> ValidateXml - add validation error as an attribute of invalid flow files
> 
>
> Key: NIFI-4278
> URL: https://issues.apache.org/jira/browse/NIFI-4278
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Minor
> Fix For: 1.4.0
>
>
> The idea is to have an attribute containing the validation error message for 
> flow files routed to the invalid relationship.
> I initially added a specific property allowing the user to decide if this 
> attribute should be added or not... but in the end I removed it as I think 
> this is a desired behaviour in any situation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4278) ValidateXml - add validation error as an attribute of invalid flow files

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121040#comment-16121040
 ] 

ASF GitHub Bot commented on NIFI-4278:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2069


> ValidateXml - add validation error as an attribute of invalid flow files
> 
>
> Key: NIFI-4278
> URL: https://issues.apache.org/jira/browse/NIFI-4278
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Minor
>
> The idea is to have an attribute containing the validation error message for 
> flow files routed to the invalid relationship.
> I initially added a specific property allowing the user to decide if this 
> attribute should be added or not... but in the end I removed it as I think 
> this is a desired behaviour in any situation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4278) ValidateXml - add validation error as an attribute of invalid flow files

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121039#comment-16121039
 ] 

ASF subversion and git services commented on NIFI-4278:
---

Commit 6ff8321cf71fb07a5d961a30b035e31c48829c91 in nifi's branch 
refs/heads/master from [~pvillard]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=6ff8321 ]

NIFI-4278 - add error message to invalid FFs in ValidateXml

This closes #2069.

Signed-off-by: Andy LoPresto 


> ValidateXml - add validation error as an attribute of invalid flow files
> 
>
> Key: NIFI-4278
> URL: https://issues.apache.org/jira/browse/NIFI-4278
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Minor
>
> The idea is to have an attribute containing the validation error message for 
> flow files routed to the invalid relationship.
> I initially added a specific property allowing the user to decide if this 
> attribute should be added or not... but in the end I removed it as I think 
> this is a desired behaviour in any situation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi pull request #2069: NIFI-4278 - add error message to invalid FFs in Val...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2069


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-4278) ValidateXml - add validation error as an attribute of invalid flow files

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121038#comment-16121038
 ] 

ASF GitHub Bot commented on NIFI-4278:
--

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2069
  
I ran this with a simple `GenFF/ValidateXML/LogAttr` flow and it looks 
great. Ran `contrib-check` and all tests pass. +1, merging. 

```
--
Standard FlowFile Attributes
Key: 'entryDate'
Value: 'Wed Aug 09 21:04:20 PDT 2017'
Key: 'lineageStartDate'
Value: 'Wed Aug 09 21:04:20 PDT 2017'
Key: 'fileSize'
Value: '24'
FlowFile Attribute Map Content
Key: 'filename'
Value: '204498845083507'
Key: 'path'
Value: './'
Key: 'uuid'
Value: 'de5ad824-cea7-442e-8f01-ca01b9aa6e06'
Key: 'validatexml.invalid.error'
Value: 'cvc-elt.1: Cannot find the declaration of element 'this'.'
--
is not valid
```


> ValidateXml - add validation error as an attribute of invalid flow files
> 
>
> Key: NIFI-4278
> URL: https://issues.apache.org/jira/browse/NIFI-4278
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Minor
>
> The idea is to have an attribute containing the validation error message for 
> flow files routed to the invalid relationship.
> I initially added a specific property allowing the user to decide if this 
> attribute should be added or not... but in the end I removed it as I think 
> this is a desired behaviour in any situation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi issue #2069: NIFI-4278 - add error message to invalid FFs in ValidateXm...

[alopresto]

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2069
  
I ran this with a simple `GenFF/ValidateXML/LogAttr` flow and it looks 
great. Ran `contrib-check` and all tests pass. +1, merging. 

```
--
Standard FlowFile Attributes
Key: 'entryDate'
Value: 'Wed Aug 09 21:04:20 PDT 2017'
Key: 'lineageStartDate'
Value: 'Wed Aug 09 21:04:20 PDT 2017'
Key: 'fileSize'
Value: '24'
FlowFile Attribute Map Content
Key: 'filename'
Value: '204498845083507'
Key: 'path'
Value: './'
Key: 'uuid'
Value: 'de5ad824-cea7-442e-8f01-ca01b9aa6e06'
Key: 'validatexml.invalid.error'
Value: 'cvc-elt.1: Cannot find the declaration of element 'this'.'
--
is not valid
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-4278) ValidateXml - add validation error as an attribute of invalid flow files

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121016#comment-16121016
 ] 

ASF GitHub Bot commented on NIFI-4278:
--

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2069
  
Reviewing...


> ValidateXml - add validation error as an attribute of invalid flow files
> 
>
> Key: NIFI-4278
> URL: https://issues.apache.org/jira/browse/NIFI-4278
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Minor
>
> The idea is to have an attribute containing the validation error message for 
> flow files routed to the invalid relationship.
> I initially added a specific property allowing the user to decide if this 
> attribute should be added or not... but in the end I removed it as I think 
> this is a desired behaviour in any situation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi issue #2069: NIFI-4278 - add error message to invalid FFs in ValidateXm...

[alopresto]

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2069
  
Reviewing...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-4274) SSLContextService keystore and truststore location property descriptors incorrectly attempt to evaluate EL

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121010#comment-16121010
 ] 

ASF GitHub Bot commented on NIFI-4274:
--

GitHub user alopresto opened a pull request:

https://github.com/apache/nifi/pull/2071

NIFI-4274 Removed incorrect EL evaluation from StandardSSLContextService 
file validator

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [x] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [x] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [x] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/alopresto/nifi NIFI-4274

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2071.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2071


commit 09e338c3e75ae5e295fc1697fb2c887e02a67785
Author: Andy LoPresto 
Date:   2017-08-10T03:03:33Z

NIFI-4274 Added unit tests for StandardSSLContextService file validator.

commit dd0d8904adf49038e2afcff0aebd477597099cf1
Author: Andy LoPresto 
Date:   2017-08-10T03:44:27Z

NIFI-4274 Added failing unit test for StandardSSLContextService file 
validator.

commit cbbba6789e39b0810e98636088625f7778ded59c
Author: Andy LoPresto 
Date:   2017-08-10T03:46:10Z

NIFI-4274 Removed EL evaluation logic from custom file validator.
All tests pass.




> SSLContextService keystore and truststore location property descriptors 
> incorrectly attempt to evaluate EL
> --
>
> Key: NIFI-4274
> URL: https://issues.apache.org/jira/browse/NIFI-4274
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.3.0
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>  Labels: expression-language, security, tls, truststore
>
> As reported on [Stack Overflow|https://stackoverflow.com/q/45561985/70465], 
> the {{StandardSSLContextService}} truststore location property descriptor 
> would not evaluate an environment variable containing the location of the 
> truststore file. The reporter said that by adding a space prior to the EL 
> expression, it would evaluate, but result in an invalid path because it 
> started with a space. 
> Bryan Bende pointed out that this field does not support Expression Language. 
> While I could not reproduce this behavior, I did verify using a remote 
> debugger that while the field does not support EL, the [custom file validator 
> incorrectly attempts to evaluate 
> EL|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-services/nifi-ssl-context-bundle/nifi-ssl-context-service/src/main/java/org/apache/nifi/ssl/StandardSSLContextService.java#L183-L183],
>  which is counter-indicated by the documentation and will cause issues. This 
> line follows immediately after comments explaining the existence of the 
> custom validator is because the default evaluates EL, which is not desired 
> here. 
> While 

[GitHub] nifi pull request #2071: NIFI-4274 Removed incorrect EL evaluation from Stan...

[alopresto]

GitHub user alopresto opened a pull request:

https://github.com/apache/nifi/pull/2071

NIFI-4274 Removed incorrect EL evaluation from StandardSSLContextService 
file validator

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [x] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [x] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [x] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/alopresto/nifi NIFI-4274

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2071.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2071


commit 09e338c3e75ae5e295fc1697fb2c887e02a67785
Author: Andy LoPresto 
Date:   2017-08-10T03:03:33Z

NIFI-4274 Added unit tests for StandardSSLContextService file validator.

commit dd0d8904adf49038e2afcff0aebd477597099cf1
Author: Andy LoPresto 
Date:   2017-08-10T03:44:27Z

NIFI-4274 Added failing unit test for StandardSSLContextService file 
validator.

commit cbbba6789e39b0810e98636088625f7778ded59c
Author: Andy LoPresto 
Date:   2017-08-10T03:46:10Z

NIFI-4274 Removed EL evaluation logic from custom file validator.
All tests pass.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (NIFI-4274) SSLContextService keystore and truststore location property descriptors incorrectly attempt to evaluate EL

[Andy LoPresto (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andy LoPresto reassigned NIFI-4274:
---

Assignee: Andy LoPresto

> SSLContextService keystore and truststore location property descriptors 
> incorrectly attempt to evaluate EL
> --
>
> Key: NIFI-4274
> URL: https://issues.apache.org/jira/browse/NIFI-4274
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.3.0
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>  Labels: expression-language, security, tls, truststore
>
> As reported on [Stack Overflow|https://stackoverflow.com/q/45561985/70465], 
> the {{StandardSSLContextService}} truststore location property descriptor 
> would not evaluate an environment variable containing the location of the 
> truststore file. The reporter said that by adding a space prior to the EL 
> expression, it would evaluate, but result in an invalid path because it 
> started with a space. 
> Bryan Bende pointed out that this field does not support Expression Language. 
> While I could not reproduce this behavior, I did verify using a remote 
> debugger that while the field does not support EL, the [custom file validator 
> incorrectly attempts to evaluate 
> EL|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-services/nifi-ssl-context-bundle/nifi-ssl-context-service/src/main/java/org/apache/nifi/ssl/StandardSSLContextService.java#L183-L183],
>  which is counter-indicated by the documentation and will cause issues. This 
> line follows immediately after comments explaining the existence of the 
> custom validator is because the default evaluates EL, which is not desired 
> here. 
> While personally, I do not believe these fields should support EL (security 
> risk of the sensitive location being changed outside of NiFi with no 
> visibility), the documentation and actual behavior should at least agree. 
> The custom validator should not evaluate EL. Follow on discussion on this 
> ticket or the mailing list may lead to new requirements to handle EL, but 
> this can be implemented correctly and consistently at such time. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4256) Add support for all AWS S3 Encryption Options

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120950#comment-16120950
 ] 

ASF GitHub Bot commented on NIFI-4256:
--

Github user jvwing commented on the issue:

https://github.com/apache/nifi/pull/2066
  
Reviewing...


> Add support for all AWS S3 Encryption Options
> -
>
> Key: NIFI-4256
> URL: https://issues.apache.org/jira/browse/NIFI-4256
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Affects Versions: 1.2.0
>Reporter: Franco
>  Labels: aws, aws-s3, security
> Fix For: 1.4.0
>
>
> NiFi currently only supports SSE-S3 encryption (AES256).
> Support needs to be added for:
> * SSE-S3
> * SSE-KMS
> * SSE-C
> * CSE-KMS CMK
> * CSE-Master Key
> With all of the appropriate configuration options and such that SSE is 
> available only for PutS3Object whilst CSE is available also for FetchS3Object.
> Given that this will add another 20 or so UI properties the intention is to 
> split it into a Client Side Encryption Service and Server Side Encryption 
> Service. This will allow users to reuse "encryption" across different 
> workflows.
> Existing flows using the Server Side Encryption option will still work as is 
> but will be overridden if a service is added.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi issue #2066: NIFI-4256 - Add support for all AWS S3 Encryption Options

[jvwing]

Github user jvwing commented on the issue:

https://github.com/apache/nifi/pull/2066
  
Reviewing...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-4222) TLS Toolkit should provide SAN by default

[Andy LoPresto (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andy LoPresto updated NIFI-4222:

   Resolution: Fixed
Fix Version/s: 1.4.0
   Status: Resolved  (was: Patch Available)

> TLS Toolkit should provide SAN by default
> -
>
> Key: NIFI-4222
> URL: https://issues.apache.org/jira/browse/NIFI-4222
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Tools and Build
>Affects Versions: 1.3.0
>Reporter: Andy LoPresto
>Assignee: Pierre Villard
>  Labels: security, tls, tls-toolkit
> Fix For: 1.4.0
>
>
> As of Chrome 58, the browser will only use the *SubjectAlternativeName* 
> entries to determine hostname verification, rather than the *CN*. This is 
> specified in RFC 6215 [1], TLS hostname verification must attempt to use the 
> SAN entries first and may only use the CN entry if no SAN entries are 
> available. 
> Chrome takes this a step further [2]: 
> {quote}
> During Transport Layer Security (TLS) connections, Chrome browser checks to 
> make sure the connection to the site is using a valid, trusted server 
> certificate.
> For Chrome 58 and later, only the subjectAlternativeName extension, not 
> commonName, is used to match the domain name and site certificate. The 
> certificate subject alternative name can be a domain name or IP address. If 
> the certificate doesn’t have the correct subjectAlternativeName extension, 
> users get a NET::ERR_CERT_COMMON_NAME_INVALID error letting them know that 
> the connection isn’t private. If the certificate is missing a 
> subjectAlternativeName extension, users see a warning in the Security panel 
> in Chrome DevTools that lets them know the subject alternative name is 
> missing.
> {quote}
> As this will cause issues for users who do not manually provide a SAN when 
> generating their certificates using the TLS Toolkit, the toolkit should be 
> modified to automatically include the provided CN as a SAN entry, in addition 
> to any manually-provided SAN entries. 
> [1] https://tools.ietf.org/html/rfc6125#section-6.4.4
> [2] https://support.google.com/chrome/a/answer/7391219?hl=en



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4222) TLS Toolkit should provide SAN by default

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120942#comment-16120942
 ] 

ASF GitHub Bot commented on NIFI-4222:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2042


> TLS Toolkit should provide SAN by default
> -
>
> Key: NIFI-4222
> URL: https://issues.apache.org/jira/browse/NIFI-4222
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Tools and Build
>Affects Versions: 1.3.0
>Reporter: Andy LoPresto
>Assignee: Pierre Villard
>  Labels: security, tls, tls-toolkit
> Fix For: 1.4.0
>
>
> As of Chrome 58, the browser will only use the *SubjectAlternativeName* 
> entries to determine hostname verification, rather than the *CN*. This is 
> specified in RFC 6215 [1], TLS hostname verification must attempt to use the 
> SAN entries first and may only use the CN entry if no SAN entries are 
> available. 
> Chrome takes this a step further [2]: 
> {quote}
> During Transport Layer Security (TLS) connections, Chrome browser checks to 
> make sure the connection to the site is using a valid, trusted server 
> certificate.
> For Chrome 58 and later, only the subjectAlternativeName extension, not 
> commonName, is used to match the domain name and site certificate. The 
> certificate subject alternative name can be a domain name or IP address. If 
> the certificate doesn’t have the correct subjectAlternativeName extension, 
> users get a NET::ERR_CERT_COMMON_NAME_INVALID error letting them know that 
> the connection isn’t private. If the certificate is missing a 
> subjectAlternativeName extension, users see a warning in the Security panel 
> in Chrome DevTools that lets them know the subject alternative name is 
> missing.
> {quote}
> As this will cause issues for users who do not manually provide a SAN when 
> generating their certificates using the TLS Toolkit, the toolkit should be 
> modified to automatically include the provided CN as a SAN entry, in addition 
> to any manually-provided SAN entries. 
> [1] https://tools.ietf.org/html/rfc6125#section-6.4.4
> [2] https://support.google.com/chrome/a/answer/7391219?hl=en



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4222) TLS Toolkit should provide SAN by default

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120941#comment-16120941
 ] 

ASF subversion and git services commented on NIFI-4222:
---

Commit 9f1267e9490084219517e4a56c7fa7fcb0d4063e in nifi's branch 
refs/heads/master from [~pvillard]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=9f1267e ]

NIFI-4222 - Adding CN by default in SANs for generated certificates with 
tls-toolkit

This closes #2042.

Signed-off-by: Andy LoPresto 


> TLS Toolkit should provide SAN by default
> -
>
> Key: NIFI-4222
> URL: https://issues.apache.org/jira/browse/NIFI-4222
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Tools and Build
>Affects Versions: 1.3.0
>Reporter: Andy LoPresto
>Assignee: Pierre Villard
>  Labels: security, tls, tls-toolkit
>
> As of Chrome 58, the browser will only use the *SubjectAlternativeName* 
> entries to determine hostname verification, rather than the *CN*. This is 
> specified in RFC 6215 [1], TLS hostname verification must attempt to use the 
> SAN entries first and may only use the CN entry if no SAN entries are 
> available. 
> Chrome takes this a step further [2]: 
> {quote}
> During Transport Layer Security (TLS) connections, Chrome browser checks to 
> make sure the connection to the site is using a valid, trusted server 
> certificate.
> For Chrome 58 and later, only the subjectAlternativeName extension, not 
> commonName, is used to match the domain name and site certificate. The 
> certificate subject alternative name can be a domain name or IP address. If 
> the certificate doesn’t have the correct subjectAlternativeName extension, 
> users get a NET::ERR_CERT_COMMON_NAME_INVALID error letting them know that 
> the connection isn’t private. If the certificate is missing a 
> subjectAlternativeName extension, users see a warning in the Security panel 
> in Chrome DevTools that lets them know the subject alternative name is 
> missing.
> {quote}
> As this will cause issues for users who do not manually provide a SAN when 
> generating their certificates using the TLS Toolkit, the toolkit should be 
> modified to automatically include the provided CN as a SAN entry, in addition 
> to any manually-provided SAN entries. 
> [1] https://tools.ietf.org/html/rfc6125#section-6.4.4
> [2] https://support.google.com/chrome/a/answer/7391219?hl=en



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi pull request #2042: NIFI-4222 - Adding CN by default in SANs for genera...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2042


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-4222) TLS Toolkit should provide SAN by default

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120937#comment-16120937
 ] 

ASF GitHub Bot commented on NIFI-4222:
--

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2042
  
Verified that all tests and contrib-check pass. When run with no SAN 
arguments, the CN is present as a SAN. When run with additional SAN arguments, 
all are present. +1, merging. 

No SAN:
```

hw12203:...assembly/target/nifi-toolkit-1.4.0-SNAPSHOT-bin/nifi-toolkit-1.4.0-SNAPSHOT
 (pr2042) alopresto
 186058s @ 18:43:33 $ ./bin/tls-toolkit.sh standalone -n 
'nifi.nifi.apache.org' -P password -S password -f 
../../../../../nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/conf/nifi.properties
2017/08/09 18:58:45 INFO [main] 
org.apache.nifi.toolkit.tls.standalone.TlsToolkitStandaloneCommandLine: Using 
../../../../../nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/conf/nifi.properties
 as template.
2017/08/09 18:58:46 INFO [main] 
org.apache.nifi.toolkit.tls.standalone.TlsToolkitStandalone: Running standalone 
certificate generation with output directory ../nifi-toolkit-1.4.0-SNAPSHOT
2017/08/09 18:58:46 INFO [main] 
org.apache.nifi.toolkit.tls.standalone.TlsToolkitStandalone: Generated new CA 
certificate ../nifi-toolkit-1.4.0-SNAPSHOT/nifi-cert.pem and key 
../nifi-toolkit-1.4.0-SNAPSHOT/nifi-key.key
2017/08/09 18:58:46 INFO [main] 
org.apache.nifi.toolkit.tls.standalone.TlsToolkitStandalone: Writing new ssl 
configuration to ../nifi-toolkit-1.4.0-SNAPSHOT/nifi.nifi.apache.org
2017/08/09 18:58:46 INFO [main] 
org.apache.nifi.toolkit.tls.standalone.TlsToolkitStandalone: Successfully 
generated TLS configuration for nifi.nifi.apache.org 1 in 
../nifi-toolkit-1.4.0-SNAPSHOT/nifi.nifi.apache.org
2017/08/09 18:58:46 INFO [main] 
org.apache.nifi.toolkit.tls.standalone.TlsToolkitStandalone: No clientCertDn 
specified, not generating any client certificates.
2017/08/09 18:58:46 INFO [main] 
org.apache.nifi.toolkit.tls.standalone.TlsToolkitStandalone: tls-toolkit 
standalone completed successfully

hw12203:...assembly/target/nifi-toolkit-1.4.0-SNAPSHOT-bin/nifi-toolkit-1.4.0-SNAPSHOT
 (pr2042) alopresto
 186980s @ 18:58:55 $ cd nifi.nifi.apache.org/

hw12203:...toolkit-1.4.0-SNAPSHOT-bin/nifi-toolkit-1.4.0-SNAPSHOT/nifi.nifi.apache.org
 (pr2042) alopresto
 186988s @ 18:59:03 $ keytool -list -v -keystore keystore.jks
Enter keystore password:

Keystore type: JKS
Keystore provider: SUN

Your keystore contains 1 entry

Alias name: nifi-key
Creation date: Aug 9, 2017
Entry type: PrivateKeyEntry
Certificate chain length: 2
Certificate[1]:
Owner: CN=nifi.nifi.apache.org, OU=NIFI
Issuer: CN=localhost, OU=NIFI
Serial number: 15dc9dd8f39
Valid from: Wed Aug 09 18:58:46 PDT 2017 until: Sat Aug 08 18:58:46 PDT 2020
Certificate fingerprints:
 MD5:  E4:E8:C4:19:C1:06:86:17:C8:E5:13:F6:6F:54:0F:AE
 SHA1: 92:6B:FD:9D:89:55:A5:48:AD:31:A3:FD:A3:A6:6C:A5:C4:A8:31:0E
 SHA256: 
54:8D:30:D2:ED:9A:B0:AE:8C:37:40:9F:2F:80:2D:4A:DC:5D:14:2E:15:57:4C:71:CF:77:D6:F0:3F:92:6D:04
 Signature algorithm name: SHA256withRSA
 Version: 3

Extensions:

#1: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
: 6B 65 AB 68 5A 0A CB 59   A2 B9 0B 9E 36 2D 60 47  ke.hZ..Y6-`G
0010: 21 08 08 25!..%
]
]

#2: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
  CA:false
  PathLen: undefined
]

#3: ObjectId: 2.5.29.37 Criticality=false
ExtendedKeyUsages [
  clientAuth
  serverAuth
]

#4: ObjectId: 2.5.29.15 Criticality=true
KeyUsage [
  DigitalSignature
  Non_repudiation
  Key_Encipherment
  Data_Encipherment
  Key_Agreement
]

#5: ObjectId: 2.5.29.17 Criticality=false
SubjectAlternativeName [
  DNSName: nifi.nifi.apache.org
]

#6: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
: D9 18 43 B3 38 24 18 89   E6 1B 62 D7 AB 35 C5 14  ..C.8$b..5..
0010: 88 E9 19 E3
]
]

Certificate[2]:
Owner: CN=localhost, OU=NIFI
Issuer: CN=localhost, OU=NIFI
Serial number: 15dc9dd8d4c
Valid from: Wed Aug 09 18:58:46 PDT 2017 until: Sat Aug 08 18:58:46 PDT 2020
Certificate fingerprints:
 MD5:  A1:9E:4A:7C:65:F1:B7:E9:8F:4D:D0:18:74:E8:AA:2E
 SHA1: CD:31:8B:74:85:C7:21:4A:DB:F6:58:34:69:B7:19:6C:3B:9E:CE:00
 SHA256: 
A9:AB:C5:73:9D:B3:ED:C3:D5:79:BD:4B:E0:14:1D:0F:DC:68:41:BC:09:70:5B:2D:BD:E0:AB:49:55:14:79:3B
 Signature algorithm name: 

[GitHub] nifi issue #2042: NIFI-4222 - Adding CN by default in SANs for generated cer...

[alopresto]

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2042
  
Verified that all tests and contrib-check pass. When run with no SAN 
arguments, the CN is present as a SAN. When run with additional SAN arguments, 
all are present. +1, merging. 

No SAN:
```

hw12203:...assembly/target/nifi-toolkit-1.4.0-SNAPSHOT-bin/nifi-toolkit-1.4.0-SNAPSHOT
 (pr2042) alopresto
🔓 186058s @ 18:43:33 $ ./bin/tls-toolkit.sh standalone -n 
'nifi.nifi.apache.org' -P password -S password -f 
../../../../../nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/conf/nifi.properties
2017/08/09 18:58:45 INFO [main] 
org.apache.nifi.toolkit.tls.standalone.TlsToolkitStandaloneCommandLine: Using 
../../../../../nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/conf/nifi.properties
 as template.
2017/08/09 18:58:46 INFO [main] 
org.apache.nifi.toolkit.tls.standalone.TlsToolkitStandalone: Running standalone 
certificate generation with output directory ../nifi-toolkit-1.4.0-SNAPSHOT
2017/08/09 18:58:46 INFO [main] 
org.apache.nifi.toolkit.tls.standalone.TlsToolkitStandalone: Generated new CA 
certificate ../nifi-toolkit-1.4.0-SNAPSHOT/nifi-cert.pem and key 
../nifi-toolkit-1.4.0-SNAPSHOT/nifi-key.key
2017/08/09 18:58:46 INFO [main] 
org.apache.nifi.toolkit.tls.standalone.TlsToolkitStandalone: Writing new ssl 
configuration to ../nifi-toolkit-1.4.0-SNAPSHOT/nifi.nifi.apache.org
2017/08/09 18:58:46 INFO [main] 
org.apache.nifi.toolkit.tls.standalone.TlsToolkitStandalone: Successfully 
generated TLS configuration for nifi.nifi.apache.org 1 in 
../nifi-toolkit-1.4.0-SNAPSHOT/nifi.nifi.apache.org
2017/08/09 18:58:46 INFO [main] 
org.apache.nifi.toolkit.tls.standalone.TlsToolkitStandalone: No clientCertDn 
specified, not generating any client certificates.
2017/08/09 18:58:46 INFO [main] 
org.apache.nifi.toolkit.tls.standalone.TlsToolkitStandalone: tls-toolkit 
standalone completed successfully

hw12203:...assembly/target/nifi-toolkit-1.4.0-SNAPSHOT-bin/nifi-toolkit-1.4.0-SNAPSHOT
 (pr2042) alopresto
🔓 186980s @ 18:58:55 $ cd nifi.nifi.apache.org/

hw12203:...toolkit-1.4.0-SNAPSHOT-bin/nifi-toolkit-1.4.0-SNAPSHOT/nifi.nifi.apache.org
 (pr2042) alopresto
🔓 186988s @ 18:59:03 $ keytool -list -v -keystore keystore.jks
Enter keystore password:

Keystore type: JKS
Keystore provider: SUN

Your keystore contains 1 entry

Alias name: nifi-key
Creation date: Aug 9, 2017
Entry type: PrivateKeyEntry
Certificate chain length: 2
Certificate[1]:
Owner: CN=nifi.nifi.apache.org, OU=NIFI
Issuer: CN=localhost, OU=NIFI
Serial number: 15dc9dd8f39
Valid from: Wed Aug 09 18:58:46 PDT 2017 until: Sat Aug 08 18:58:46 PDT 2020
Certificate fingerprints:
 MD5:  E4:E8:C4:19:C1:06:86:17:C8:E5:13:F6:6F:54:0F:AE
 SHA1: 92:6B:FD:9D:89:55:A5:48:AD:31:A3:FD:A3:A6:6C:A5:C4:A8:31:0E
 SHA256: 
54:8D:30:D2:ED:9A:B0:AE:8C:37:40:9F:2F:80:2D:4A:DC:5D:14:2E:15:57:4C:71:CF:77:D6:F0:3F:92:6D:04
 Signature algorithm name: SHA256withRSA
 Version: 3

Extensions:

#1: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
: 6B 65 AB 68 5A 0A CB 59   A2 B9 0B 9E 36 2D 60 47  ke.hZ..Y6-`G
0010: 21 08 08 25!..%
]
]

#2: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
  CA:false
  PathLen: undefined
]

#3: ObjectId: 2.5.29.37 Criticality=false
ExtendedKeyUsages [
  clientAuth
  serverAuth
]

#4: ObjectId: 2.5.29.15 Criticality=true
KeyUsage [
  DigitalSignature
  Non_repudiation
  Key_Encipherment
  Data_Encipherment
  Key_Agreement
]

#5: ObjectId: 2.5.29.17 Criticality=false
SubjectAlternativeName [
  DNSName: nifi.nifi.apache.org
]

#6: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
: D9 18 43 B3 38 24 18 89   E6 1B 62 D7 AB 35 C5 14  ..C.8$b..5..
0010: 88 E9 19 E3
]
]

Certificate[2]:
Owner: CN=localhost, OU=NIFI
Issuer: CN=localhost, OU=NIFI
Serial number: 15dc9dd8d4c
Valid from: Wed Aug 09 18:58:46 PDT 2017 until: Sat Aug 08 18:58:46 PDT 2020
Certificate fingerprints:
 MD5:  A1:9E:4A:7C:65:F1:B7:E9:8F:4D:D0:18:74:E8:AA:2E
 SHA1: CD:31:8B:74:85:C7:21:4A:DB:F6:58:34:69:B7:19:6C:3B:9E:CE:00
 SHA256: 
A9:AB:C5:73:9D:B3:ED:C3:D5:79:BD:4B:E0:14:1D:0F:DC:68:41:BC:09:70:5B:2D:BD:E0:AB:49:55:14:79:3B
 Signature algorithm name: SHA256withRSA
 Version: 3

Extensions:

#1: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
: 6B 65 AB 68 5A 0A CB 59   A2 B9 0B 9E 36 2D 60 47  

[jira] [Commented] (NIFI-4222) TLS Toolkit should provide SAN by default

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120888#comment-16120888
 ] 

ASF GitHub Bot commented on NIFI-4222:
--

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2042
  
Reviewing...


> TLS Toolkit should provide SAN by default
> -
>
> Key: NIFI-4222
> URL: https://issues.apache.org/jira/browse/NIFI-4222
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Tools and Build
>Affects Versions: 1.3.0
>Reporter: Andy LoPresto
>Assignee: Pierre Villard
>  Labels: security, tls, tls-toolkit
>
> As of Chrome 58, the browser will only use the *SubjectAlternativeName* 
> entries to determine hostname verification, rather than the *CN*. This is 
> specified in RFC 6215 [1], TLS hostname verification must attempt to use the 
> SAN entries first and may only use the CN entry if no SAN entries are 
> available. 
> Chrome takes this a step further [2]: 
> {quote}
> During Transport Layer Security (TLS) connections, Chrome browser checks to 
> make sure the connection to the site is using a valid, trusted server 
> certificate.
> For Chrome 58 and later, only the subjectAlternativeName extension, not 
> commonName, is used to match the domain name and site certificate. The 
> certificate subject alternative name can be a domain name or IP address. If 
> the certificate doesn’t have the correct subjectAlternativeName extension, 
> users get a NET::ERR_CERT_COMMON_NAME_INVALID error letting them know that 
> the connection isn’t private. If the certificate is missing a 
> subjectAlternativeName extension, users see a warning in the Security panel 
> in Chrome DevTools that lets them know the subject alternative name is 
> missing.
> {quote}
> As this will cause issues for users who do not manually provide a SAN when 
> generating their certificates using the TLS Toolkit, the toolkit should be 
> modified to automatically include the provided CN as a SAN entry, in addition 
> to any manually-provided SAN entries. 
> [1] https://tools.ietf.org/html/rfc6125#section-6.4.4
> [2] https://support.google.com/chrome/a/answer/7391219?hl=en



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi issue #2042: NIFI-4222 - Adding CN by default in SANs for generated cer...

[alopresto]

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2042
  
Reviewing...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-4210) Add OpenId Connect support for authenticating users

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120851#comment-16120851
 ] 

ASF GitHub Bot commented on NIFI-4210:
--

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2047
  
I continued reviewing with a custom IdP from Auth0. This provider can 
integrate with other providers, including Google, Facebook, etc. and also have 
its own custom users stored locally. @mcgilman and I were able to receive a 
success response from Auth0 for a custom user, but the token signature could 
not be validated. We traced this to determine that on startup, when the 
`TokenValidator` is initialized, if no 
`nifi.security.user.oidc.preferred.jwsalgorithm` is present in 
`nifi.properties`, the first value of `id_token_signing_alg_values_supported` 
in the response from the OpenID Connect Discovery Document URI is used. In 
Auth0's case, this was `HS256` as opposed to `RS256`, but the tokens returned 
were signed with `RS256`, resulting in NiFi returning an error "Unable to 
exchange authorization for ID token: Unable to parse the response from the 
Token request: Signed JWT rejected: Another algorithm expected, or no matching 
key(s) found". 

The Auth0 OIDC Discovery Document response:

```

{"issuer":"https://apachenifi.auth0.com/","authorization_endpoint":"https://apachenifi.auth0.com/authorize","token_endpoint":"https://apachenifi.auth0.com/oauth/token","userinfo_endpoint":"https://apachenifi.auth0.com/userinfo","mfa_challenge_endpoint":"https://apachenifi.auth0.com/mfa/challenge","jwks_uri":"https://apachenifi.auth0.com/.well-known/jwks.json","registration_endpoint":"https://apachenifi.auth0.com/oidc/register","scopes_supported":["openid","profile","offline_access","name","given_name","family_name","nickname","email","email_verified","picture","created_at","identities","phone","address"],"response_types_supported":["code","token","id_token","code
 token","code id_token","token id_token","code token 
id_token"],"response_modes_supported":["query","fragment","form_post"],"subject_types_supported":["public"],"id_token_signing_alg_values_supported":["HS256","RS256"],"token_endpoint_auth_methods_supported":["client_secret_basic","client_secret_post"],"claims_supported":["aud","auth_time","created_at","email","email_verified","exp","family_name","given_name","iat","identities","iss","name","nickname","phone_number","picture","sub"]}
```

We decided to change the initialization logic to default to `RS256` as it 
is required by the [OpenID Connect 
Specification](https://openid.net/specs/openid-connect-discovery-1_0.html#ProviderMetadata)
 to always be present in this response. 

> id_token_signing_alg_values_supported
> REQUIRED. JSON array containing a list of the JWS signing algorithms (alg 
values) supported by the OP for the ID Token to encode the Claims in a JWT 
[JWT]. **The algorithm RS256 MUST be included**. The value none MAY be 
supported, but MUST NOT be used unless the Response Type used returns no ID 
Token from the Authorization Endpoint (such as when using the Authorization 
Code Flow).

I manually provided `RS256` as a configuration value for 
`nifi.security.user.oidc.preferred.jwsalgorithm` in `nifi.properties` and was 
able to successfully sign in as the user. 


> Add OpenId Connect support for authenticating users
> ---
>
> Key: NIFI-4210
> URL: https://issues.apache.org/jira/browse/NIFI-4210
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework, Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>
> Add support for authenticating users with the OpenId Connection 
> specification. Evaluate whether a new extension point is necessary to allow 
> for a given provider to supply custom code for instance to implement custom 
> token validation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi issue #2047: NIFI-4210: Add support for OpenId Connect

[alopresto]

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2047
  
I continued reviewing with a custom IdP from Auth0. This provider can 
integrate with other providers, including Google, Facebook, etc. and also have 
its own custom users stored locally. @mcgilman and I were able to receive a 
success response from Auth0 for a custom user, but the token signature could 
not be validated. We traced this to determine that on startup, when the 
`TokenValidator` is initialized, if no 
`nifi.security.user.oidc.preferred.jwsalgorithm` is present in 
`nifi.properties`, the first value of `id_token_signing_alg_values_supported` 
in the response from the OpenID Connect Discovery Document URI is used. In 
Auth0's case, this was `HS256` as opposed to `RS256`, but the tokens returned 
were signed with `RS256`, resulting in NiFi returning an error "Unable to 
exchange authorization for ID token: Unable to parse the response from the 
Token request: Signed JWT rejected: Another algorithm expected, or no matching 
key(s) found". 

The Auth0 OIDC Discovery Document response:

```

{"issuer":"https://apachenifi.auth0.com/","authorization_endpoint":"https://apachenifi.auth0.com/authorize","token_endpoint":"https://apachenifi.auth0.com/oauth/token","userinfo_endpoint":"https://apachenifi.auth0.com/userinfo","mfa_challenge_endpoint":"https://apachenifi.auth0.com/mfa/challenge","jwks_uri":"https://apachenifi.auth0.com/.well-known/jwks.json","registration_endpoint":"https://apachenifi.auth0.com/oidc/register","scopes_supported":["openid","profile","offline_access","name","given_name","family_name","nickname","email","email_verified","picture","created_at","identities","phone","address"],"response_types_supported":["code","token","id_token","code
 token","code id_token","token id_token","code token 
id_token"],"response_modes_supported":["query","fragment","form_post"],"subject_types_supported":["public"],"id_token_signing_alg_values_supported":["HS256","RS256"],"token_endpoint_auth_methods_supported":["client_secret_basic","client_secret_post"],"claims_supported":
 
["aud","auth_time","created_at","email","email_verified","exp","family_name","given_name","iat","identities","iss","name","nickname","phone_number","picture","sub"]}
```

We decided to change the initialization logic to default to `RS256` as it 
is required by the [OpenID Connect 
Specification](https://openid.net/specs/openid-connect-discovery-1_0.html#ProviderMetadata)
 to always be present in this response. 

> id_token_signing_alg_values_supported
> REQUIRED. JSON array containing a list of the JWS signing algorithms (alg 
values) supported by the OP for the ID Token to encode the Claims in a JWT 
[JWT]. **The algorithm RS256 MUST be included**. The value none MAY be 
supported, but MUST NOT be used unless the Response Type used returns no ID 
Token from the Authorization Endpoint (such as when using the Authorization 
Code Flow).

I manually provided `RS256` as a configuration value for 
`nifi.security.user.oidc.preferred.jwsalgorithm` in `nifi.properties` and was 
able to successfully sign in as the user. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2162) InvokeHttp's underlying library for Digest Auth uses the Android logger

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120720#comment-16120720
 ] 

ASF GitHub Bot commented on NIFI-2162:
--

Github user JPercivall commented on the issue:

https://github.com/apache/nifi/pull/2004
  
Thanks for the review @trkurc and sorry for the delay in my responses.


> InvokeHttp's underlying library for Digest Auth uses the Android logger
> ---
>
> Key: NIFI-2162
> URL: https://issues.apache.org/jira/browse/NIFI-2162
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joseph Percivall
>Assignee: Joseph Percivall
>
> A user emailed the User mailing list with an issue that InvokeHttp was 
> failing due to not being able to find "android/util/Log"[1]. InvokeHttp uses 
> OkHttp and the library they recommend for digest authentication is 
> okhttp-digest[2]. Currently okhttp-digest assumes it's running on an Android 
> device and has access to the Android logger (OkHttp does not assume it's on 
> an Android device). 
> I raised an issue about it on the project's github page[3] and the creator 
> said he "Will change this soonish."
> Once that is addressed, InvokeHttp will need to update the versions of OkHttp 
> and okhttp-digest. 
> [1] http://mail-archives.apache.org/mod_mbox/nifi-users/201606.mbox/browser
> [2] https://github.com/square/okhttp/issues/205
> [3] https://github.com/rburgst/okhttp-digest/issues/13



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi issue #2004: NIFI-2162 Updating OkHttp to 3.8.1 and OkHttp-Digest to 1....

[JPercivall]

Github user JPercivall commented on the issue:

https://github.com/apache/nifi/pull/2004
  
Thanks for the review @trkurc and sorry for the delay in my responses.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2162) InvokeHttp's underlying library for Digest Auth uses the Android logger

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120719#comment-16120719
 ] 

ASF GitHub Bot commented on NIFI-2162:
--

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2004#discussion_r132318693
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/InvokeHTTP.java
 ---
@@ -500,48 +512,88 @@ public void onPropertyModified(final 
PropertyDescriptor descriptor, final String
 }
 
 @OnScheduled
-public void setUpClient(final ProcessContext context) throws 
IOException {
+public void setUpClient(final ProcessContext context) throws 
IOException, UnrecoverableKeyException, CertificateException, 
NoSuchAlgorithmException, KeyStoreException, KeyManagementException {
 okHttpClientAtomicReference.set(null);
 
-OkHttpClient okHttpClient = new OkHttpClient();
+OkHttpClient.Builder okHttpClientBuilder = new 
OkHttpClient().newBuilder();
 
 // Add a proxy if set
 final String proxyHost = 
context.getProperty(PROP_PROXY_HOST).getValue();
 final Integer proxyPort = 
context.getProperty(PROP_PROXY_PORT).asInteger();
 if (proxyHost != null && proxyPort != null) {
 final Proxy proxy = new Proxy(Type.HTTP, new 
InetSocketAddress(proxyHost, proxyPort));
-okHttpClient.setProxy(proxy);
+okHttpClientBuilder.proxy(proxy);
 }
 
 // Set timeouts
-
okHttpClient.setConnectTimeout((context.getProperty(PROP_CONNECT_TIMEOUT).asTimePeriod(TimeUnit.MILLISECONDS).intValue()),
 TimeUnit.MILLISECONDS);
-
okHttpClient.setReadTimeout(context.getProperty(PROP_READ_TIMEOUT).asTimePeriod(TimeUnit.MILLISECONDS).intValue(),
 TimeUnit.MILLISECONDS);
+
okHttpClientBuilder.connectTimeout((context.getProperty(PROP_CONNECT_TIMEOUT).asTimePeriod(TimeUnit.MILLISECONDS).intValue()),
 TimeUnit.MILLISECONDS);
+
okHttpClientBuilder.readTimeout(context.getProperty(PROP_READ_TIMEOUT).asTimePeriod(TimeUnit.MILLISECONDS).intValue(),
 TimeUnit.MILLISECONDS);
 
 // Set whether to follow redirects
-
okHttpClient.setFollowRedirects(context.getProperty(PROP_FOLLOW_REDIRECTS).asBoolean());
+
okHttpClientBuilder.followRedirects(context.getProperty(PROP_FOLLOW_REDIRECTS).asBoolean());
 
 final SSLContextService sslService = 
context.getProperty(PROP_SSL_CONTEXT_SERVICE).asControllerService(SSLContextService.class);
 final SSLContext sslContext = sslService == null ? null : 
sslService.createSSLContext(ClientAuth.NONE);
 
 // check if the ssl context is set and add the factory if so
 if (sslContext != null) {
-
okHttpClient.setSslSocketFactory(sslContext.getSocketFactory());
+setSslSocketFactory(okHttpClientBuilder, sslService, 
sslContext);
 }
 
 // check the trusted hostname property and override the 
HostnameVerifier
 String trustedHostname = 
trimToEmpty(context.getProperty(PROP_TRUSTED_HOSTNAME).getValue());
 if (!trustedHostname.isEmpty()) {
-okHttpClient.setHostnameVerifier(new 
OverrideHostnameVerifier(trustedHostname, okHttpClient.getHostnameVerifier()));
+okHttpClientBuilder.hostnameVerifier(new 
OverrideHostnameVerifier(trustedHostname, OkHostnameVerifier.INSTANCE));
 }
 
-setAuthenticator(okHttpClient, context);
+setAuthenticator(okHttpClientBuilder, context);
 
 useChunked = 
context.getProperty(PROP_USE_CHUNKED_ENCODING).asBoolean();
 
-okHttpClientAtomicReference.set(okHttpClient);
+okHttpClientAtomicReference.set(okHttpClientBuilder.build());
+}
+
+private void setSslSocketFactory(OkHttpClient.Builder 
okHttpClientBuilder, SSLContextService sslService, SSLContext sslContext)
+throws IOException, KeyStoreException, CertificateException, 
NoSuchAlgorithmException, UnrecoverableKeyException, KeyManagementException {
+final String keystoreLocation = sslService.getKeyStoreFile();
+final String keystorePass = sslService.getKeyStorePassword();
+final String keystoreType = sslService.getKeyStoreType();
+
+// prepare the keystore
+final KeyStore keyStore = KeyStore.getInstance(keystoreType);
+
+try (FileInputStream keyStoreStream = new 
FileInputStream(keystoreLocation)) {
+keyStore.load(keyStoreStream, keystorePass.toCharArray());
+}
+
+final KeyManagerFactory keyManagerFactory = 

[GitHub] nifi pull request #2004: NIFI-2162 Updating OkHttp to 3.8.1 and OkHttp-Diges...

[JPercivall]

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2004#discussion_r132318693
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/InvokeHTTP.java
 ---
@@ -500,48 +512,88 @@ public void onPropertyModified(final 
PropertyDescriptor descriptor, final String
 }
 
 @OnScheduled
-public void setUpClient(final ProcessContext context) throws 
IOException {
+public void setUpClient(final ProcessContext context) throws 
IOException, UnrecoverableKeyException, CertificateException, 
NoSuchAlgorithmException, KeyStoreException, KeyManagementException {
 okHttpClientAtomicReference.set(null);
 
-OkHttpClient okHttpClient = new OkHttpClient();
+OkHttpClient.Builder okHttpClientBuilder = new 
OkHttpClient().newBuilder();
 
 // Add a proxy if set
 final String proxyHost = 
context.getProperty(PROP_PROXY_HOST).getValue();
 final Integer proxyPort = 
context.getProperty(PROP_PROXY_PORT).asInteger();
 if (proxyHost != null && proxyPort != null) {
 final Proxy proxy = new Proxy(Type.HTTP, new 
InetSocketAddress(proxyHost, proxyPort));
-okHttpClient.setProxy(proxy);
+okHttpClientBuilder.proxy(proxy);
 }
 
 // Set timeouts
-
okHttpClient.setConnectTimeout((context.getProperty(PROP_CONNECT_TIMEOUT).asTimePeriod(TimeUnit.MILLISECONDS).intValue()),
 TimeUnit.MILLISECONDS);
-
okHttpClient.setReadTimeout(context.getProperty(PROP_READ_TIMEOUT).asTimePeriod(TimeUnit.MILLISECONDS).intValue(),
 TimeUnit.MILLISECONDS);
+
okHttpClientBuilder.connectTimeout((context.getProperty(PROP_CONNECT_TIMEOUT).asTimePeriod(TimeUnit.MILLISECONDS).intValue()),
 TimeUnit.MILLISECONDS);
+
okHttpClientBuilder.readTimeout(context.getProperty(PROP_READ_TIMEOUT).asTimePeriod(TimeUnit.MILLISECONDS).intValue(),
 TimeUnit.MILLISECONDS);
 
 // Set whether to follow redirects
-
okHttpClient.setFollowRedirects(context.getProperty(PROP_FOLLOW_REDIRECTS).asBoolean());
+
okHttpClientBuilder.followRedirects(context.getProperty(PROP_FOLLOW_REDIRECTS).asBoolean());
 
 final SSLContextService sslService = 
context.getProperty(PROP_SSL_CONTEXT_SERVICE).asControllerService(SSLContextService.class);
 final SSLContext sslContext = sslService == null ? null : 
sslService.createSSLContext(ClientAuth.NONE);
 
 // check if the ssl context is set and add the factory if so
 if (sslContext != null) {
-
okHttpClient.setSslSocketFactory(sslContext.getSocketFactory());
+setSslSocketFactory(okHttpClientBuilder, sslService, 
sslContext);
 }
 
 // check the trusted hostname property and override the 
HostnameVerifier
 String trustedHostname = 
trimToEmpty(context.getProperty(PROP_TRUSTED_HOSTNAME).getValue());
 if (!trustedHostname.isEmpty()) {
-okHttpClient.setHostnameVerifier(new 
OverrideHostnameVerifier(trustedHostname, okHttpClient.getHostnameVerifier()));
+okHttpClientBuilder.hostnameVerifier(new 
OverrideHostnameVerifier(trustedHostname, OkHostnameVerifier.INSTANCE));
 }
 
-setAuthenticator(okHttpClient, context);
+setAuthenticator(okHttpClientBuilder, context);
 
 useChunked = 
context.getProperty(PROP_USE_CHUNKED_ENCODING).asBoolean();
 
-okHttpClientAtomicReference.set(okHttpClient);
+okHttpClientAtomicReference.set(okHttpClientBuilder.build());
+}
+
+private void setSslSocketFactory(OkHttpClient.Builder 
okHttpClientBuilder, SSLContextService sslService, SSLContext sslContext)
+throws IOException, KeyStoreException, CertificateException, 
NoSuchAlgorithmException, UnrecoverableKeyException, KeyManagementException {
+final String keystoreLocation = sslService.getKeyStoreFile();
+final String keystorePass = sslService.getKeyStorePassword();
+final String keystoreType = sslService.getKeyStoreType();
+
+// prepare the keystore
+final KeyStore keyStore = KeyStore.getInstance(keystoreType);
+
+try (FileInputStream keyStoreStream = new 
FileInputStream(keystoreLocation)) {
+keyStore.load(keyStoreStream, keystorePass.toCharArray());
+}
+
+final KeyManagerFactory keyManagerFactory = 
KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
+keyManagerFactory.init(keyStore, keystorePass.toCharArray());
+
+// load truststore
+final String truststoreLocation = sslService.getTrustStoreFile();
+

[jira] [Commented] (NIFI-2162) InvokeHttp's underlying library for Digest Auth uses the Android logger

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120718#comment-16120718
 ] 

ASF GitHub Bot commented on NIFI-2162:
--

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2004#discussion_r132318357
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/ProxyAuthenticator.java
 ---
@@ -16,35 +16,28 @@
  */
 package org.apache.nifi.processors.standard.util;
 
+import okhttp3.Authenticator;
+import okhttp3.Credentials;
+import okhttp3.Request;
+import okhttp3.Response;
+import okhttp3.Route;
+
+import javax.annotation.Nullable;
 import java.io.IOException;
-import java.net.Proxy;
-import java.util.HashMap;
-import java.util.Map;
 
-import com.burgstaller.okhttp.DispatchingAuthenticator;
-import com.squareup.okhttp.Authenticator;
-import com.squareup.okhttp.Credentials;
-import com.squareup.okhttp.Request;
-import com.squareup.okhttp.Response;
+public class ProxyAuthenticator implements Authenticator {
--- End diff --

I was targeting 1.4.0

I understand that it's a public class but I wouldn't consider it part of 
the public API. Looking at the items explicitly listed that fall under the 
public API, the closest one I see is "Any extension such as Processor, 
Controller Service, Reporting Task." I see it falling outside of that though 
since it's not a component itself. In what way do you see it falling under the 
items listed as our public API?


> InvokeHttp's underlying library for Digest Auth uses the Android logger
> ---
>
> Key: NIFI-2162
> URL: https://issues.apache.org/jira/browse/NIFI-2162
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joseph Percivall
>Assignee: Joseph Percivall
>
> A user emailed the User mailing list with an issue that InvokeHttp was 
> failing due to not being able to find "android/util/Log"[1]. InvokeHttp uses 
> OkHttp and the library they recommend for digest authentication is 
> okhttp-digest[2]. Currently okhttp-digest assumes it's running on an Android 
> device and has access to the Android logger (OkHttp does not assume it's on 
> an Android device). 
> I raised an issue about it on the project's github page[3] and the creator 
> said he "Will change this soonish."
> Once that is addressed, InvokeHttp will need to update the versions of OkHttp 
> and okhttp-digest. 
> [1] http://mail-archives.apache.org/mod_mbox/nifi-users/201606.mbox/browser
> [2] https://github.com/square/okhttp/issues/205
> [3] https://github.com/rburgst/okhttp-digest/issues/13



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi pull request #2004: NIFI-2162 Updating OkHttp to 3.8.1 and OkHttp-Diges...

[JPercivall]

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2004#discussion_r132318357
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/ProxyAuthenticator.java
 ---
@@ -16,35 +16,28 @@
  */
 package org.apache.nifi.processors.standard.util;
 
+import okhttp3.Authenticator;
+import okhttp3.Credentials;
+import okhttp3.Request;
+import okhttp3.Response;
+import okhttp3.Route;
+
+import javax.annotation.Nullable;
 import java.io.IOException;
-import java.net.Proxy;
-import java.util.HashMap;
-import java.util.Map;
 
-import com.burgstaller.okhttp.DispatchingAuthenticator;
-import com.squareup.okhttp.Authenticator;
-import com.squareup.okhttp.Credentials;
-import com.squareup.okhttp.Request;
-import com.squareup.okhttp.Response;
+public class ProxyAuthenticator implements Authenticator {
--- End diff --

I was targeting 1.4.0

I understand that it's a public class but I wouldn't consider it part of 
the public API. Looking at the items explicitly listed that fall under the 
public API, the closest one I see is "Any extension such as Processor, 
Controller Service, Reporting Task." I see it falling outside of that though 
since it's not a component itself. In what way do you see it falling under the 
items listed as our public API?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi-cpp pull request #126: MINIFI-350 added pytest-based system inte...

[achristianson]

GitHub user achristianson opened a pull request:

https://github.com/apache/nifi-minifi-cpp/pull/126

MINIFI-350 added pytest-based system integration test framework and initial 
test cases

Added a test framework which enables automated tests suits for the docker 
image as well as simulation/system integration testing of multiple-host flows 
and/or integration with other non-MiNiFi components.

Currently, the HTTP tests are failing due to 
https://issues.apache.org/jira/browse/MINIFI-369, but once the fix for that is 
merged, the test suite will pass.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/achristianson/nifi-minifi-cpp MINIFI-350

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi-cpp/pull/126.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #126


commit a9b4be93cf2cec21b314854776c412f7d0a2282c
Author: Andrew Christianson 
Date:   2017-07-13T14:42:35Z

MINIFI-350 added pytest-based system integration test framework and initial 
test cases




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (NIFIREG-8) Define Web REST API

[Bryan Bende (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFIREG-8?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Bende resolved NIFIREG-8.
---
   Resolution: Fixed
Fix Version/s: 0.0.1

> Define Web REST API 
> 
>
> Key: NIFIREG-8
> URL: https://issues.apache.org/jira/browse/NIFIREG-8
> Project: NiFi Registry
>  Issue Type: New Feature
>Reporter: Kevin Doran
>Assignee: Kevin Doran
> Fix For: 0.0.1
>
>
> Start to define a  REST Web API that will serve as the contract between 
> clients (e.g., NiFi) and the web service.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFIREG-8) Define Web REST API

[Bryan Bende (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFIREG-8?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120435#comment-16120435
 ] 

Bryan Bende commented on NIFIREG-8:
---

Merged to master.

> Define Web REST API 
> 
>
> Key: NIFIREG-8
> URL: https://issues.apache.org/jira/browse/NIFIREG-8
> Project: NiFi Registry
>  Issue Type: New Feature
>Reporter: Kevin Doran
>Assignee: Kevin Doran
> Fix For: 0.0.1
>
>
> Start to define a  REST Web API that will serve as the contract between 
> clients (e.g., NiFi) and the web service.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (NIFIREG-8) Define Web REST API

[Bryan Bende (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFIREG-8?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Bende reassigned NIFIREG-8:
-

Assignee: Kevin Doran

> Define Web REST API 
> 
>
> Key: NIFIREG-8
> URL: https://issues.apache.org/jira/browse/NIFIREG-8
> Project: NiFi Registry
>  Issue Type: New Feature
>Reporter: Kevin Doran
>Assignee: Kevin Doran
> Fix For: 0.0.1
>
>
> Start to define a  REST Web API that will serve as the contract between 
> clients (e.g., NiFi) and the web service.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFIREG-8) Define Web REST API

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFIREG-8?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120403#comment-16120403
 ] 

ASF GitHub Bot commented on NIFIREG-8:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-registry/pull/4


> Define Web REST API 
> 
>
> Key: NIFIREG-8
> URL: https://issues.apache.org/jira/browse/NIFIREG-8
> Project: NiFi Registry
>  Issue Type: New Feature
>Reporter: Kevin Doran
>
> Start to define a  REST Web API that will serve as the contract between 
> clients (e.g., NiFi) and the web service.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi-registry pull request #4: NIFIREG-8: Registry Web API Stubs

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-registry/pull/4


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-4275) Allow specifying timestamp in PutHBase processors

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120374#comment-16120374
 ] 

ASF GitHub Bot commented on NIFI-4275:
--

GitHub user bbende opened a pull request:

https://github.com/apache/nifi/pull/2070

NIFI-4275 Adding support for specifying the timestamp on PutHBase pro…

…cessors




You can merge this pull request into a Git repository by running:

$ git pull https://github.com/bbende/nifi NIFI-4275

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2070.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2070


commit 73a3ffc650a4a3a6936757ce8f9f2310eee3bf5c
Author: Bryan Bende 
Date:   2017-08-09T18:01:19Z

NIFI-4275 Adding support for specifying the timestamp on PutHBase processors




> Allow specifying timestamp in PutHBase processors
> -
>
> Key: NIFI-4275
> URL: https://issues.apache.org/jira/browse/NIFI-4275
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Bryan Bende
>Assignee: Bryan Bende
>Priority: Minor
>
> We should add the ability to specify the timestamp of a cell in PutHBase 
> processors.
> * PutHBaseCell - Add an optional property called "Timestamp" that supports 
> EL, if specified it would get passed to the Put operation sent to HBase
> * PutHBaseJson - Add an optional property called "Timestamp" that supports 
> EL, if specified this would be the timestamp used for all cells created from 
> the JSON document
> * PutHBaseRecord - Add an optional property called "Timestamp Field Name" 
> that supports EL, if specified the value of this field would be used as the 
> timestamp of each cell



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (NIFI-4275) Allow specifying timestamp in PutHBase processors

[Bryan Bende (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Bende updated NIFI-4275:
--
Status: Patch Available  (was: In Progress)

> Allow specifying timestamp in PutHBase processors
> -
>
> Key: NIFI-4275
> URL: https://issues.apache.org/jira/browse/NIFI-4275
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Bryan Bende
>Assignee: Bryan Bende
>Priority: Minor
>
> We should add the ability to specify the timestamp of a cell in PutHBase 
> processors.
> * PutHBaseCell - Add an optional property called "Timestamp" that supports 
> EL, if specified it would get passed to the Put operation sent to HBase
> * PutHBaseJson - Add an optional property called "Timestamp" that supports 
> EL, if specified this would be the timestamp used for all cells created from 
> the JSON document
> * PutHBaseRecord - Add an optional property called "Timestamp Field Name" 
> that supports EL, if specified the value of this field would be used as the 
> timestamp of each cell



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi pull request #2070: NIFI-4275 Adding support for specifying the timesta...

[bbende]

GitHub user bbende opened a pull request:

https://github.com/apache/nifi/pull/2070

NIFI-4275 Adding support for specifying the timestamp on PutHBase pro…

…cessors




You can merge this pull request into a Git repository by running:

$ git pull https://github.com/bbende/nifi NIFI-4275

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2070.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2070


commit 73a3ffc650a4a3a6936757ce8f9f2310eee3bf5c
Author: Bryan Bende 
Date:   2017-08-09T18:01:19Z

NIFI-4275 Adding support for specifying the timestamp on PutHBase processors




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120360#comment-16120360
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r132257748
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -238,6 +313,47 @@ public Response getProcessGroup(
 return generateOkResponse(entity).build();
 }
 
+
+/**
+ * Retrieves the Variable Registry for the group with the given ID
+ *
+ * @param groupId the ID of the Process Group
+ * @return the Variable Registry for the group
+ */
+@GET
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{id}/variable-registry")
+@ApiOperation(value = "Gets a process group's variable registry", 
response = VariableRegistryEntity.class, authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = "")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response getVariableRegistry(
--- End diff --

Can we add a flag (probably enabled by default) that will include all 
variables available in this scope (defined in this Process Group and any 
ancestors)? Also, the response should indicate where each variable is defined.


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120361#comment-16120361
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r132257197
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/ProcessGroupDTO.java
 ---
@@ -200,4 +204,13 @@ public void setInactiveRemotePortCount(Integer 
inactiveRemotePortCount) {
 this.inactiveRemotePortCount = inactiveRemotePortCount;
 }
 
+
+@ApiModelProperty("The variables that are configured for the Process 
Group")
+public Map getVariables() {
+return variables;
+}
+
+public void setVariables(final Map variables) {
--- End diff --

Also, can we update the documentation to indicate that this only includes 
the variables defined at this Process Group?


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

[mcgilman]

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r132257748
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -238,6 +313,47 @@ public Response getProcessGroup(
 return generateOkResponse(entity).build();
 }
 
+
+/**
+ * Retrieves the Variable Registry for the group with the given ID
+ *
+ * @param groupId the ID of the Process Group
+ * @return the Variable Registry for the group
+ */
+@GET
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{id}/variable-registry")
+@ApiOperation(value = "Gets a process group's variable registry", 
response = VariableRegistryEntity.class, authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = "")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response getVariableRegistry(
--- End diff --

Can we add a flag (probably enabled by default) that will include all 
variables available in this scope (defined in this Process Group and any 
ancestors)? Also, the response should indicate where each variable is defined.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

[mcgilman]

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r132257197
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/ProcessGroupDTO.java
 ---
@@ -200,4 +204,13 @@ public void setInactiveRemotePortCount(Integer 
inactiveRemotePortCount) {
 this.inactiveRemotePortCount = inactiveRemotePortCount;
 }
 
+
+@ApiModelProperty("The variables that are configured for the Process 
Group")
+public Map getVariables() {
+return variables;
+}
+
+public void setVariables(final Map variables) {
--- End diff --

Also, can we update the documentation to indicate that this only includes 
the variables defined at this Process Group?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi-cpp pull request #125: MINIFI-368 exclude hidden files when scan...

[achristianson]

GitHub user achristianson opened a pull request:

https://github.com/apache/nifi-minifi-cpp/pull/125

MINIFI-368 exclude hidden files when scanning for src files

Updated regex matching when scanning for source files so that we don't pick 
up hidden files such as vim swap files.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/achristianson/nifi-minifi-cpp MINIFI-368

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi-cpp/pull/125.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #125


commit d3dd216c208b43a10aa086a5463ff906b430c490
Author: Andrew I. Christianson 
Date:   2017-08-09T15:53:04Z

MINIFI-368 exclude hidden files when scanning for src files




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-4255) Add support for providing ACLs for paths in Zookeeper Migration tool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120043#comment-16120043
 ] 

ASF GitHub Bot commented on NIFI-4255:
--

Github user jtstorck commented on the issue:

https://github.com/apache/nifi/pull/2065
  
Reviewing...


> Add support for providing ACLs for paths in Zookeeper Migration tool
> 
>
> Key: NIFI-4255
> URL: https://issues.apache.org/jira/browse/NIFI-4255
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Tools and Build
>Affects Versions: 1.3.0
>Reporter: Yolanda M. Davis
>Assignee: Yolanda M. Davis
>
> Currently in the Zookeeper migration utility there is support for applying 
> acls when importing zookeeper data (Znodes).  However this support only 
> applies default ACLs values (either Open or Creator specific), and the value 
> used depends on if security is enabled or disabled in the destination 
> Zookeeper instance. This may become problematic if the user/identity used to 
> import zookeeper data does not align with the users/identities that require 
> read/modify rights on the imported Znodes. This also doesn't provide users 
> flexibility in defining specific rights or applying additional authorizations 
> on paths.
> Enhancing the existing utility to support providing ACL information would 
> offer users more flexibility in defining permissions and authentication 
> schemes on znodes. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi issue #2065: NIFI-4255 - added flag to allow migration of existing (sou...

[jtstorck]

Github user jtstorck commented on the issue:

https://github.com/apache/nifi/pull/2065
  
Reviewing...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-3335) GenerateTableFetch should allow you to specify an initial Max Value

[Matt Burgess (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Burgess updated NIFI-3335:
---
Issue Type: Improvement  (was: New Feature)

> GenerateTableFetch should allow you to specify an initial Max Value
> ---
>
> Key: NIFI-3335
> URL: https://issues.apache.org/jira/browse/NIFI-3335
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Matt Burgess
>Assignee: Matt Burgess
> Fix For: 1.4.0
>
>
> NIFI-2583 added the ability (via dynamic properties) to specify initial Max 
> Values for columns, to enable the user to "pick up where they left off" if 
> something happened with a flow, a NiFi instance, etc. where the state was 
> stored but the processing did not complete successfully.
> This feature would also be helpful in GenerateTableFetch, which also supports 
> max-value columns.
> Since NIFI-2881 adds incoming flow file support, it's more useful if Initial 
> max values can be specified via flow file attributes. Because if a table name 
> is dynamically passed via flow file attribute and Expression Language, user 
> won't be able to configure dynamic processor attribute in advance for each 
> possible table.
> Add dynamic properties ('initial.maxvalue.' same as 
> QueryDatabaseTable) to specify initial max values statically, and also use 
> incoming flow file attributes named 'initial.maxvalue.' if 
> any. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (NIFI-3335) GenerateTableFetch should allow you to specify an initial Max Value

[Matt Burgess (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Burgess reassigned NIFI-3335:
--

Assignee: Matt Burgess

> GenerateTableFetch should allow you to specify an initial Max Value
> ---
>
> Key: NIFI-3335
> URL: https://issues.apache.org/jira/browse/NIFI-3335
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Reporter: Matt Burgess
>Assignee: Matt Burgess
> Fix For: 1.4.0
>
>
> NIFI-2583 added the ability (via dynamic properties) to specify initial Max 
> Values for columns, to enable the user to "pick up where they left off" if 
> something happened with a flow, a NiFi instance, etc. where the state was 
> stored but the processing did not complete successfully.
> This feature would also be helpful in GenerateTableFetch, which also supports 
> max-value columns.
> Since NIFI-2881 adds incoming flow file support, it's more useful if Initial 
> max values can be specified via flow file attributes. Because if a table name 
> is dynamically passed via flow file attribute and Expression Language, user 
> won't be able to configure dynamic processor attribute in advance for each 
> possible table.
> Add dynamic properties ('initial.maxvalue.' same as 
> QueryDatabaseTable) to specify initial max values statically, and also use 
> incoming flow file attributes named 'initial.maxvalue.' if 
> any. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (NIFI-3335) GenerateTableFetch should allow you to specify an initial Max Value

[Matt Burgess (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Burgess reassigned NIFI-3335:
--

Assignee: (was: Matt Burgess)

> GenerateTableFetch should allow you to specify an initial Max Value
> ---
>
> Key: NIFI-3335
> URL: https://issues.apache.org/jira/browse/NIFI-3335
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Reporter: Matt Burgess
> Fix For: 1.4.0
>
>
> NIFI-2583 added the ability (via dynamic properties) to specify initial Max 
> Values for columns, to enable the user to "pick up where they left off" if 
> something happened with a flow, a NiFi instance, etc. where the state was 
> stored but the processing did not complete successfully.
> This feature would also be helpful in GenerateTableFetch, which also supports 
> max-value columns.
> Since NIFI-2881 adds incoming flow file support, it's more useful if Initial 
> max values can be specified via flow file attributes. Because if a table name 
> is dynamically passed via flow file attribute and Expression Language, user 
> won't be able to configure dynamic processor attribute in advance for each 
> possible table.
> Add dynamic properties ('initial.maxvalue.' same as 
> QueryDatabaseTable) to specify initial max values statically, and also use 
> incoming flow file attributes named 'initial.maxvalue.' if 
> any. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (NIFI-3335) GenerateTableFetch should allow you to specify an initial Max Value

[Matt Burgess (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Burgess updated NIFI-3335:
---
Issue Type: New Feature  (was: Improvement)

> GenerateTableFetch should allow you to specify an initial Max Value
> ---
>
> Key: NIFI-3335
> URL: https://issues.apache.org/jira/browse/NIFI-3335
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Reporter: Matt Burgess
>Assignee: Matt Burgess
> Fix For: 1.4.0
>
>
> NIFI-2583 added the ability (via dynamic properties) to specify initial Max 
> Values for columns, to enable the user to "pick up where they left off" if 
> something happened with a flow, a NiFi instance, etc. where the state was 
> stored but the processing did not complete successfully.
> This feature would also be helpful in GenerateTableFetch, which also supports 
> max-value columns.
> Since NIFI-2881 adds incoming flow file support, it's more useful if Initial 
> max values can be specified via flow file attributes. Because if a table name 
> is dynamically passed via flow file attribute and Expression Language, user 
> won't be able to configure dynamic processor attribute in advance for each 
> possible table.
> Add dynamic properties ('initial.maxvalue.' same as 
> QueryDatabaseTable) to specify initial max values statically, and also use 
> incoming flow file attributes named 'initial.maxvalue.' if 
> any. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFIREG-8) Define Web REST API

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFIREG-8?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119967#comment-16119967
 ] 

ASF GitHub Bot commented on NIFIREG-8:
--

Github user kevdoran commented on the issue:

https://github.com/apache/nifi-registry/pull/4
  
@bbende Thanks for reviewing this!

That's odd regarding the swagger plugin. Thanks for doing a contrib-check, 
I did not realize that was setup for this sub-project yet. I'm good with you 
making those changes on merge. 

Thanks again!


> Define Web REST API 
> 
>
> Key: NIFIREG-8
> URL: https://issues.apache.org/jira/browse/NIFIREG-8
> Project: NiFi Registry
>  Issue Type: New Feature
>Reporter: Kevin Doran
>
> Start to define a  REST Web API that will serve as the contract between 
> clients (e.g., NiFi) and the web service.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi-registry issue #4: NIFIREG-8: Registry Web API Stubs

[kevdoran]

Github user kevdoran commented on the issue:

https://github.com/apache/nifi-registry/pull/4
  
@bbende Thanks for reviewing this!

That's odd regarding the swagger plugin. Thanks for doing a contrib-check, 
I did not realize that was setup for this sub-project yet. I'm good with you 
making those changes on merge. 

Thanks again!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi-cpp pull request #124: MINIFI-367 port tests to use boost::files...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi-cpp/pull/124


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (NIFI-4275) Allow specifying timestamp in PutHBase processors

[Bryan Bende (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Bende reassigned NIFI-4275:
-

Assignee: Bryan Bende

> Allow specifying timestamp in PutHBase processors
> -
>
> Key: NIFI-4275
> URL: https://issues.apache.org/jira/browse/NIFI-4275
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Bryan Bende
>Assignee: Bryan Bende
>Priority: Minor
>
> We should add the ability to specify the timestamp of a cell in PutHBase 
> processors.
> * PutHBaseCell - Add an optional property called "Timestamp" that supports 
> EL, if specified it would get passed to the Put operation sent to HBase
> * PutHBaseJson - Add an optional property called "Timestamp" that supports 
> EL, if specified this would be the timestamp used for all cells created from 
> the JSON document
> * PutHBaseRecord - Add an optional property called "Timestamp Field Name" 
> that supports EL, if specified the value of this field would be used as the 
> timestamp of each cell



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4210) Add OpenId Connect support for authenticating users

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119840#comment-16119840
 ] 

ASF GitHub Bot commented on NIFI-4210:
--

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/2047
  
@alopresto This is not something that was introduced in this PR though I am 
happy to address it here. Will push a new commit shortly. Thanks!


> Add OpenId Connect support for authenticating users
> ---
>
> Key: NIFI-4210
> URL: https://issues.apache.org/jira/browse/NIFI-4210
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework, Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>
> Add support for authenticating users with the OpenId Connection 
> specification. Evaluate whether a new extension point is necessary to allow 
> for a given provider to supply custom code for instance to implement custom 
> token validation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi issue #2047: NIFI-4210: Add support for OpenId Connect

[mcgilman]

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/2047
  
@alopresto This is not something that was introduced in this PR though I am 
happy to address it here. Will push a new commit shortly. Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-4192) Issue with the MergeContent Processor when processing Avro files

[Pierre Villard (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pierre Villard updated NIFI-4192:
-
Component/s: Extensions

> Issue with the MergeContent Processor when processing Avro files
> 
>
> Key: NIFI-4192
> URL: https://issues.apache.org/jira/browse/NIFI-4192
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Matt Burgess
>Assignee: Matt Burgess
> Fix For: 1.4.0
>
>
> Currently the Avro Merge strategy in MergeContent requires that any key/value 
> pairs in the Avro non-reserved metadata must match. This might have been done 
> to support re-merging of split Avro files upstream, since the "fragment.*" 
> capability was added afterward.
> Now that the fragment.* attributes are available to help merge a batch of 
> flow files, the user should be able to select the Avro Metadata Strategy the 
> same way as the Attribute Strategy, with the additional option of "Ignore if 
> unmatched", which should be default to maintain currently functionality.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (NIFI-4192) Issue with the MergeContent Processor when processing Avro files

[Pierre Villard (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pierre Villard updated NIFI-4192:
-
   Resolution: Fixed
Fix Version/s: 1.4.0
   Status: Resolved  (was: Patch Available)

> Issue with the MergeContent Processor when processing Avro files
> 
>
> Key: NIFI-4192
> URL: https://issues.apache.org/jira/browse/NIFI-4192
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Matt Burgess
>Assignee: Matt Burgess
> Fix For: 1.4.0
>
>
> Currently the Avro Merge strategy in MergeContent requires that any key/value 
> pairs in the Avro non-reserved metadata must match. This might have been done 
> to support re-merging of split Avro files upstream, since the "fragment.*" 
> capability was added afterward.
> Now that the fragment.* attributes are available to help merge a batch of 
> flow files, the user should be able to select the Avro Metadata Strategy the 
> same way as the Attribute Strategy, with the additional option of "Ignore if 
> unmatched", which should be default to maintain currently functionality.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi pull request #2067: NIFI-4192: Add more Avro/metadata merge options to ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2067


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-4192) Issue with the MergeContent Processor when processing Avro files

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119715#comment-16119715
 ] 

ASF GitHub Bot commented on NIFI-4192:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2067


> Issue with the MergeContent Processor when processing Avro files
> 
>
> Key: NIFI-4192
> URL: https://issues.apache.org/jira/browse/NIFI-4192
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>
> Currently the Avro Merge strategy in MergeContent requires that any key/value 
> pairs in the Avro non-reserved metadata must match. This might have been done 
> to support re-merging of split Avro files upstream, since the "fragment.*" 
> capability was added afterward.
> Now that the fragment.* attributes are available to help merge a batch of 
> flow files, the user should be able to select the Avro Metadata Strategy the 
> same way as the Attribute Strategy, with the additional option of "Ignore if 
> unmatched", which should be default to maintain currently functionality.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4192) Issue with the MergeContent Processor when processing Avro files

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119714#comment-16119714
 ] 

ASF subversion and git services commented on NIFI-4192:
---

Commit 9c4fdd4ef3589d64e4e15822fc9fdb79a2315535 in nifi's branch 
refs/heads/master from [~mattyb149]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=9c4fdd4 ]

NIFI-4192: Add more Avro/metadata merge options to MergeContent

Signed-off-by: Pierre Villard 

This closes #2067.


> Issue with the MergeContent Processor when processing Avro files
> 
>
> Key: NIFI-4192
> URL: https://issues.apache.org/jira/browse/NIFI-4192
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>
> Currently the Avro Merge strategy in MergeContent requires that any key/value 
> pairs in the Avro non-reserved metadata must match. This might have been done 
> to support re-merging of split Avro files upstream, since the "fragment.*" 
> capability was added afterward.
> Now that the fragment.* attributes are available to help merge a batch of 
> flow files, the user should be able to select the Avro Metadata Strategy the 
> same way as the Attribute Strategy, with the additional option of "Ignore if 
> unmatched", which should be default to maintain currently functionality.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4278) ValidateXml - add validation error as an attribute of invalid flow files

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119692#comment-16119692
 ] 

ASF GitHub Bot commented on NIFI-4278:
--

GitHub user pvillard31 opened a pull request:

https://github.com/apache/nifi/pull/2069

NIFI-4278 - add error message to invalid FFs in ValidateXml

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/pvillard31/nifi NIFI-4278

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2069.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2069


commit 9762c8813c188b9c0714aa47867cc8f1cff9de8c
Author: Pierre Villard 
Date:   2017-08-09T10:34:24Z

NIFI-4278 - add error message to invalid FFs in ValidateXml




> ValidateXml - add validation error as an attribute of invalid flow files
> 
>
> Key: NIFI-4278
> URL: https://issues.apache.org/jira/browse/NIFI-4278
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Minor
>
> The idea is to have an attribute containing the validation error message for 
> flow files routed to the invalid relationship.
> I initially added a specific property allowing the user to decide if this 
> attribute should be added or not... but in the end I removed it as I think 
> this is a desired behaviour in any situation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (NIFI-4278) ValidateXml - add validation error as an attribute of invalid flow files

[Pierre Villard (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pierre Villard updated NIFI-4278:
-
Status: Patch Available  (was: Open)

> ValidateXml - add validation error as an attribute of invalid flow files
> 
>
> Key: NIFI-4278
> URL: https://issues.apache.org/jira/browse/NIFI-4278
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Minor
>
> The idea is to have an attribute containing the validation error message for 
> flow files routed to the invalid relationship.
> I initially added a specific property allowing the user to decide if this 
> attribute should be added or not... but in the end I removed it as I think 
> this is a desired behaviour in any situation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi pull request #2069: NIFI-4278 - add error message to invalid FFs in Val...

[pvillard31]

GitHub user pvillard31 opened a pull request:

https://github.com/apache/nifi/pull/2069

NIFI-4278 - add error message to invalid FFs in ValidateXml

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/pvillard31/nifi NIFI-4278

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2069.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2069


commit 9762c8813c188b9c0714aa47867cc8f1cff9de8c
Author: Pierre Villard 
Date:   2017-08-09T10:34:24Z

NIFI-4278 - add error message to invalid FFs in ValidateXml




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (NIFI-4278) ValidateXml - add validation error as an attribute of invalid flow files

[Pierre Villard (JIRA)]

Pierre Villard created NIFI-4278:


 Summary: ValidateXml - add validation error as an attribute of 
invalid flow files
 Key: NIFI-4278
 URL: https://issues.apache.org/jira/browse/NIFI-4278
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Extensions
Reporter: Pierre Villard
Assignee: Pierre Villard
Priority: Minor


The idea is to have an attribute containing the validation error message for 
flow files routed to the invalid relationship.

I initially added a specific property allowing the user to decide if this 
attribute should be added or not... but in the end I removed it as I think this 
is a desired behaviour in any situation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (NIFI-4277) StandardLogRepository does not log exceptions

[Pierre Villard (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pierre Villard updated NIFI-4277:
-
Status: Patch Available  (was: Open)

> StandardLogRepository does not log exceptions
> -
>
> Key: NIFI-4277
> URL: https://issues.apache.org/jira/browse/NIFI-4277
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.3.0
>Reporter: Pierre Villard
>Assignee: Pierre Villard
> Attachments: Screen Shot 2017-08-08 at 2.48.33 PM.png
>
>
> When logging a message, it is logged with the SLF4J logger and also stored in 
> the standard log repository (for the bulletins). However if the array of 
> objects contains the exception (and not the message of the exception), this 
> exception won't be displayed in the bulletin message.
> That's because of:
> {code:title=StandardLogRepository.java|borderStyle=solid}
> @Override
> public void addLogMessage(final LogLevel level, final String format, 
> final Object[] params) {
> final String formattedMessage = MessageFormatter.arrayFormat(format, 
> params).getMessage();
> addLogMessage(level, formattedMessage);
> }
> {code}
> If the params object contains a Throwable object, it'll be removed from the 
> array in the {{MessageFormatter}}:
> {code:title=MessageFormatter.java|borderStyle=solid}
> final public static FormattingTuple arrayFormat(final String 
> messagePattern, final Object[] argArray) {
> Throwable throwableCandidate = getThrowableCandidate(argArray);
> Object[] args = argArray;
> if (throwableCandidate != null) {
> args = trimmedCopy(argArray);
> }
> return arrayFormat(messagePattern, args, throwableCandidate);
> }
> {code}
> Easy solution would be to change:
> {noformat}
> logger.debug("Failed to validate {} against schema due to {}", new 
> Object[]{flowFile, e});
> {noformat}
> into:
> {noformat}
> logger.debug("Failed to validate {} against schema due to {}", new 
> Object[]{flowFile, e.getLocalizedMessage()});
> {noformat}
> However this pattern can be found in quite a large number of places... And 
> it'd be certainly better to provide a permanent solution supporting the 
> existing pattern. Suggestion is to modify the method in 
> {{StandardLogRepository}} to go through all the items of the array and for 
> each Throwable object, replace it by the localized message.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4277) StandardLogRepository does not log exceptions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119637#comment-16119637
 ] 

ASF GitHub Bot commented on NIFI-4277:
--

GitHub user pvillard31 opened a pull request:

https://github.com/apache/nifi/pull/2068

NIFI-4277 Fixed exception logging in StandardLogRepository

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/pvillard31/nifi NIFI-4277

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2068.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2068


commit 7e91f4ed597d337e0e8b882d89b9d92a8a316df8
Author: Pierre Villard 
Date:   2017-08-09T09:50:53Z

NIFI-4277 Fixed exception logging in StandardLogRepository




> StandardLogRepository does not log exceptions
> -
>
> Key: NIFI-4277
> URL: https://issues.apache.org/jira/browse/NIFI-4277
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.3.0
>Reporter: Pierre Villard
>Assignee: Pierre Villard
> Attachments: Screen Shot 2017-08-08 at 2.48.33 PM.png
>
>
> When logging a message, it is logged with the SLF4J logger and also stored in 
> the standard log repository (for the bulletins). However if the array of 
> objects contains the exception (and not the message of the exception), this 
> exception won't be displayed in the bulletin message.
> That's because of:
> {code:title=StandardLogRepository.java|borderStyle=solid}
> @Override
> public void addLogMessage(final LogLevel level, final String format, 
> final Object[] params) {
> final String formattedMessage = MessageFormatter.arrayFormat(format, 
> params).getMessage();
> addLogMessage(level, formattedMessage);
> }
> {code}
> If the params object contains a Throwable object, it'll be removed from the 
> array in the {{MessageFormatter}}:
> {code:title=MessageFormatter.java|borderStyle=solid}
> final public static FormattingTuple arrayFormat(final String 
> messagePattern, final Object[] argArray) {
> Throwable throwableCandidate = getThrowableCandidate(argArray);
> Object[] args = argArray;
> if (throwableCandidate != null) {
> args = trimmedCopy(argArray);
> }
> return arrayFormat(messagePattern, args, throwableCandidate);
> }
> {code}
> Easy solution would be to change:
> {noformat}
> logger.debug("Failed to validate {} against schema due to {}", new 
> Object[]{flowFile, e});
> {noformat}
> into:
> {noformat}
> logger.debug("Failed to validate {} against schema due to {}", new 
> Object[]{flowFile, e.getLocalizedMessage()});
> {noformat}
> However this pattern can be found in quite a large number of places... And 
> it'd be certainly better to provide a permanent solution supporting the 
> existing pattern. 

[GitHub] nifi pull request #2068: NIFI-4277 Fixed exception logging in StandardLogRep...

[pvillard31]

GitHub user pvillard31 opened a pull request:

https://github.com/apache/nifi/pull/2068

NIFI-4277 Fixed exception logging in StandardLogRepository

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/pvillard31/nifi NIFI-4277

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2068.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2068


commit 7e91f4ed597d337e0e8b882d89b9d92a8a316df8
Author: Pierre Villard 
Date:   2017-08-09T09:50:53Z

NIFI-4277 Fixed exception logging in StandardLogRepository




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-4277) StandardLogRepository does not log exceptions

[Pierre Villard (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pierre Villard updated NIFI-4277:
-
Description: 
When logging a message, it is logged with the SLF4J logger and also stored in 
the standard log repository (for the bulletins). However if the array of 
objects contains the exception (and not the message of the exception), this 
exception won't be displayed in the bulletin message.

That's because of:

{code:title=StandardLogRepository.java|borderStyle=solid}
@Override
public void addLogMessage(final LogLevel level, final String format, final 
Object[] params) {
final String formattedMessage = MessageFormatter.arrayFormat(format, 
params).getMessage();
addLogMessage(level, formattedMessage);
}
{code}

If the params object contains a Throwable object, it'll be removed from the 
array in the {{MessageFormatter}}:


{code:title=MessageFormatter.java|borderStyle=solid}
final public static FormattingTuple arrayFormat(final String 
messagePattern, final Object[] argArray) {
Throwable throwableCandidate = getThrowableCandidate(argArray);
Object[] args = argArray;
if (throwableCandidate != null) {
args = trimmedCopy(argArray);
}
return arrayFormat(messagePattern, args, throwableCandidate);
}
{code}

Easy solution would be to change:

{noformat}
logger.debug("Failed to validate {} against schema due to {}", new 
Object[]{flowFile, e});
{noformat}

into:

{noformat}
logger.debug("Failed to validate {} against schema due to {}", new 
Object[]{flowFile, e.getLocalizedMessage()});
{noformat}

However this pattern can be found in quite a large number of places... And it'd 
be certainly better to provide a permanent solution supporting the existing 
pattern. Suggestion is to modify the method in {{StandardLogRepository}} to go 
through all the items of the array and for each Throwable object, replace it by 
the localized message.

  was:
When logging a message, it is logged with the SLF4J logger and also stored in 
the standard log repository (for the bulletins). However if the array of 
objects contains the exception (and not the message of the exception), this 
exception won't be displayed in the bulletin message.

!attachment-name.jpg|thumbnail!

That's because of:

{code:title=StandardLogRepository.java|borderStyle=solid}
@Override
public void addLogMessage(final LogLevel level, final String format, final 
Object[] params) {
final String formattedMessage = MessageFormatter.arrayFormat(format, 
params).getMessage();
addLogMessage(level, formattedMessage);
}
{code}

If the params object contains a Throwable object, it'll be removed from the 
array in the {{MessageFormatter}}:


{code:title=MessageFormatter.java|borderStyle=solid}
final public static FormattingTuple arrayFormat(final String 
messagePattern, final Object[] argArray) {
Throwable throwableCandidate = getThrowableCandidate(argArray);
Object[] args = argArray;
if (throwableCandidate != null) {
args = trimmedCopy(argArray);
}
return arrayFormat(messagePattern, args, throwableCandidate);
}
{code}

Easy solution would be to change:

{noformat}
logger.debug("Failed to validate {} against schema due to {}", new 
Object[]{flowFile, e});
{noformat}

into:

{noformat}
logger.debug("Failed to validate {} against schema due to {}", new 
Object[]{flowFile, e.getLocalizedMessage()});
{noformat}

However this pattern can be found in quite a large number of places... And it'd 
be certainly better to provide a permanent solution supporting the existing 
pattern. Suggestion is to modify the method in {{StandardLogRepository}} to go 
through all the items of the array and for each Throwable object, replace it by 
the localized message.


> StandardLogRepository does not log exceptions
> -
>
> Key: NIFI-4277
> URL: https://issues.apache.org/jira/browse/NIFI-4277
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.3.0
>Reporter: Pierre Villard
>Assignee: Pierre Villard
> Attachments: Screen Shot 2017-08-08 at 2.48.33 PM.png
>
>
> When logging a message, it is logged with the SLF4J logger and also stored in 
> the standard log repository (for the bulletins). However if the array of 
> objects contains the exception (and not the message of the exception), this 
> exception won't be displayed in the bulletin message.
> That's because of:
> {code:title=StandardLogRepository.java|borderStyle=solid}
> @Override
> public void addLogMessage(final LogLevel level, final String format, 
> final Object[] params) {
> final String formattedMessage = MessageFormatter.arrayFormat(format, 
> params).getMessage();
>   

[jira] [Updated] (NIFI-4277) StandardLogRepository does not log exceptions

[Pierre Villard (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pierre Villard updated NIFI-4277:
-
Description: 
When logging a message, it is logged with the SLF4J logger and also stored in 
the standard log repository (for the bulletins). However if the array of 
objects contains the exception (and not the message of the exception), this 
exception won't be displayed in the bulletin message.

!attachment-name.jpg|thumbnail!

That's because of:

{code:title=StandardLogRepository.java|borderStyle=solid}
@Override
public void addLogMessage(final LogLevel level, final String format, final 
Object[] params) {
final String formattedMessage = MessageFormatter.arrayFormat(format, 
params).getMessage();
addLogMessage(level, formattedMessage);
}
{code}

If the params object contains a Throwable object, it'll be removed from the 
array in the {{MessageFormatter}}:


{code:title=MessageFormatter.java|borderStyle=solid}
final public static FormattingTuple arrayFormat(final String 
messagePattern, final Object[] argArray) {
Throwable throwableCandidate = getThrowableCandidate(argArray);
Object[] args = argArray;
if (throwableCandidate != null) {
args = trimmedCopy(argArray);
}
return arrayFormat(messagePattern, args, throwableCandidate);
}
{code}

Easy solution would be to change:

{noformat}
logger.debug("Failed to validate {} against schema due to {}", new 
Object[]{flowFile, e});
{noformat}

into:

{noformat}
logger.debug("Failed to validate {} against schema due to {}", new 
Object[]{flowFile, e.getLocalizedMessage()});
{noformat}

However this pattern can be found in quite a large number of places... And it'd 
be certainly better to provide a permanent solution supporting the existing 
pattern. Suggestion is to modify the method in {{StandardLogRepository}} to go 
through all the items of the array and for each Throwable object, replace it by 
the localized message.

  was:
When logging a message, it is logged with the SLF4J logger and also stored in 
the standard log repository (for the bulletins). However if the array of 
objects contains the exception (and not the message of the exception), this 
exception won't be displayed in the bulletin message.

!Screen Shot 2017-08-08 at 2.48.33 PM.png|thumbnail!

That's because of:

{code:title=StandardLogRepository.java|borderStyle=solid}
@Override
public void addLogMessage(final LogLevel level, final String format, final 
Object[] params) {
final String formattedMessage = MessageFormatter.arrayFormat(format, 
params).getMessage();
addLogMessage(level, formattedMessage);
}
{code}

If the params object contains a Throwable object, it'll be removed from the 
array in the {{MessageFormatter}}:


{code:title=MessageFormatter.java|borderStyle=solid}
final public static FormattingTuple arrayFormat(final String 
messagePattern, final Object[] argArray) {
Throwable throwableCandidate = getThrowableCandidate(argArray);
Object[] args = argArray;
if (throwableCandidate != null) {
args = trimmedCopy(argArray);
}
return arrayFormat(messagePattern, args, throwableCandidate);
}
{code}

Easy solution would be to change:

{noformat}
logger.debug("Failed to validate {} against schema due to {}", new 
Object[]{flowFile, e});
{noformat}

into:

{noformat}
logger.debug("Failed to validate {} against schema due to {}", new 
Object[]{flowFile, e.getLocalizedMessage()});
{noformat}

However this pattern can be found in quite a large number of places... And it'd 
be certainly better to provide a permanent solution supporting the existing 
pattern. Suggestion is to modify the method in {{StandardLogRepository}} to go 
through all the items of the array and for each Throwable object, replace it by 
the localized message.


> StandardLogRepository does not log exceptions
> -
>
> Key: NIFI-4277
> URL: https://issues.apache.org/jira/browse/NIFI-4277
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.3.0
>Reporter: Pierre Villard
>Assignee: Pierre Villard
> Attachments: Screen Shot 2017-08-08 at 2.48.33 PM.png
>
>
> When logging a message, it is logged with the SLF4J logger and also stored in 
> the standard log repository (for the bulletins). However if the array of 
> objects contains the exception (and not the message of the exception), this 
> exception won't be displayed in the bulletin message.
> !attachment-name.jpg|thumbnail!
> That's because of:
> {code:title=StandardLogRepository.java|borderStyle=solid}
> @Override
> public void addLogMessage(final LogLevel level, final String format, 
> final Object[] params) {
> final 

[jira] [Created] (NIFI-4277) StandardLogRepository does not log exceptions

[Pierre Villard (JIRA)]

Pierre Villard created NIFI-4277:


 Summary: StandardLogRepository does not log exceptions
 Key: NIFI-4277
 URL: https://issues.apache.org/jira/browse/NIFI-4277
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Affects Versions: 1.3.0
Reporter: Pierre Villard
Assignee: Pierre Villard
 Attachments: Screen Shot 2017-08-08 at 2.48.33 PM.png

When logging a message, it is logged with the SLF4J logger and also stored in 
the standard log repository (for the bulletins). However if the array of 
objects contains the exception (and not the message of the exception), this 
exception won't be displayed in the bulletin message.

!Screen Shot 2017-08-08 at 2.48.33 PM.png|thumbnail!

That's because of:

{code:title=StandardLogRepository.java|borderStyle=solid}
@Override
public void addLogMessage(final LogLevel level, final String format, final 
Object[] params) {
final String formattedMessage = MessageFormatter.arrayFormat(format, 
params).getMessage();
addLogMessage(level, formattedMessage);
}
{code}

If the params object contains a Throwable object, it'll be removed from the 
array in the {{MessageFormatter}}:


{code:title=MessageFormatter.java|borderStyle=solid}
final public static FormattingTuple arrayFormat(final String 
messagePattern, final Object[] argArray) {
Throwable throwableCandidate = getThrowableCandidate(argArray);
Object[] args = argArray;
if (throwableCandidate != null) {
args = trimmedCopy(argArray);
}
return arrayFormat(messagePattern, args, throwableCandidate);
}
{code}

Easy solution would be to change:

{noformat}
logger.debug("Failed to validate {} against schema due to {}", new 
Object[]{flowFile, e});
{noformat}

into:

{noformat}
logger.debug("Failed to validate {} against schema due to {}", new 
Object[]{flowFile, e.getLocalizedMessage()});
{noformat}

However this pattern can be found in quite a large number of places... And it'd 
be certainly better to provide a permanent solution supporting the existing 
pattern. Suggestion is to modify the method in {{StandardLogRepository}} to go 
through all the items of the array and for each Throwable object, replace it by 
the localized message.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)