[GitHub] nifi-minifi-cpp issue #18: MINIFI-34 - attempt to progress the CMake environ...

[apiri]

Github user apiri commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/18
  
Sounds fair to me.  Indeed we have some obstacles to conquer handling all 
the varied environments but as you mentioned, the lessons learned should be a 
great help to folks who wish to build on their machine.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi-cpp issue #18: MINIFI-34 - attempt to progress the CMake environ...

[trixpan]

Github user trixpan commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/18
  
ok. So precise(12.04) does not work out of the box, only with add on 
packages. 

I think we should push this with 14.04 (travis' `dist: tahr`) and add 12.04 
as a docker container for testing. This should also help documenting the 
required compilation steps on 12.04 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi-cpp issue #18: MINIFI-34 - attempt to progress the CMake environ...

[trixpan]

Github user trixpan commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/18
  
@apiri indeed. solving the INTERFACE was a matter of using what seems to be 
called the "infamous `dummy.cpp `".

Good news is that this should be working now (it works locally).

once you confirm travis-ci precise is working, I will be happy to proceed 
with enabling trusty tahr again and setup a docker based test to ease cross 
platform development.

Major changes of the last commit

- the CMake has been adjusted to compile against existing libuuid
- travis adjusted to use whatever version of libboost-all-dev is available



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-401) New Scheduling strategy (On primary node - CRON )

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556462#comment-15556462
 ] 

ASF GitHub Bot commented on NIFI-401:
-

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/512
  
@beugley remember you must push the rebased version to github as well


> New Scheduling strategy (On primary node - CRON )
> -
>
> Key: NIFI-401
> URL: https://issues.apache.org/jira/browse/NIFI-401
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Matthew Clarke
>Priority: Minor
> Attachments: initial prototype .png
>
>
> Currently the only scheduling strategy supported when a processor is set to 
> use "On primary Node" is Timer Driven.   There should be a second option to 
> allow cron driven On primary Node scheduling strategy.  This would allow 
> users to more control over when a given primary node only processor runs. 
> This would prevent these processors from running when configuration changes 
> or instance restarts occur.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-401) New Scheduling strategy (On primary node - CRON )

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556460#comment-15556460
 ] 

ASF GitHub Bot commented on NIFI-401:
-

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/512
  
@beugley doesn't seem to have worked yet


> New Scheduling strategy (On primary node - CRON )
> -
>
> Key: NIFI-401
> URL: https://issues.apache.org/jira/browse/NIFI-401
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Matthew Clarke
>Priority: Minor
> Attachments: initial prototype .png
>
>
> Currently the only scheduling strategy supported when a processor is set to 
> use "On primary Node" is Timer Driven.   There should be a second option to 
> allow cron driven On primary Node scheduling strategy.  This would allow 
> users to more control over when a given primary node only processor runs. 
> This would prevent these processors from running when configuration changes 
> or instance restarts occur.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #512: NIFI-401

[trixpan]

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/512
  
@beugley doesn't seem to have worked yet


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-401) New Scheduling strategy (On primary node - CRON )

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556455#comment-15556455
 ] 

ASF GitHub Bot commented on NIFI-401:
-

Github user beugley commented on the issue:

https://github.com/apache/nifi/pull/512
  
Never did a rebase before.  Tried it, but not sure it worked.

From: trixpan 
Sent: Monday, October 3, 2016 7:51 PM
To: apache/nifi 
Cc: Brian Eugley ; Mention 
Subject: Re: [apache/nifi] NIFI-401 (#512)

@beugley would you mind rebasing this commit?

Cheers

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.



> New Scheduling strategy (On primary node - CRON )
> -
>
> Key: NIFI-401
> URL: https://issues.apache.org/jira/browse/NIFI-401
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Matthew Clarke
>Priority: Minor
> Attachments: initial prototype .png
>
>
> Currently the only scheduling strategy supported when a processor is set to 
> use "On primary Node" is Timer Driven.   There should be a second option to 
> allow cron driven On primary Node scheduling strategy.  This would allow 
> users to more control over when a given primary node only processor runs. 
> This would prevent these processors from running when configuration changes 
> or instance restarts occur.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #512: NIFI-401

[beugley]

Github user beugley commented on the issue:

https://github.com/apache/nifi/pull/512
  
Never did a rebase before.  Tried it, but not sure it worked.

From: trixpan 
Sent: Monday, October 3, 2016 7:51 PM
To: apache/nifi 
Cc: Brian Eugley ; Mention 
Subject: Re: [apache/nifi] NIFI-401 (#512)

@beugley would you mind rebasing this commit?

Cheers

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-401) New Scheduling strategy (On primary node - CRON )

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556442#comment-15556442
 ] 

ASF GitHub Bot commented on NIFI-401:
-

GitHub user beugley opened a pull request:

https://github.com/apache/nifi/pull/1117

NIFI-401

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/beugley/nifi master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1117.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1117


commit 011b5e724648d3f72746b18e17e79be0ebee200d
Author: Brian Eugley 
Date:   2016-06-09T15:32:15Z

NIFI-401




> New Scheduling strategy (On primary node - CRON )
> -
>
> Key: NIFI-401
> URL: https://issues.apache.org/jira/browse/NIFI-401
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Matthew Clarke
>Priority: Minor
> Attachments: initial prototype .png
>
>
> Currently the only scheduling strategy supported when a processor is set to 
> use "On primary Node" is Timer Driven.   There should be a second option to 
> allow cron driven On primary Node scheduling strategy.  This would allow 
> users to more control over when a given primary node only processor runs. 
> This would prevent these processors from running when configuration changes 
> or instance restarts occur.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1117: NIFI-401

[beugley]

GitHub user beugley opened a pull request:

https://github.com/apache/nifi/pull/1117

NIFI-401

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/beugley/nifi master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1117.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1117


commit 011b5e724648d3f72746b18e17e79be0ebee200d
Author: Brian Eugley 
Date:   2016-06-09T15:32:15Z

NIFI-401




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-1614) Simple Username/Password Authentication

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556327#comment-15556327
 ] 

ASF GitHub Bot commented on NIFI-1614:
--

Github user jvwing closed the pull request at:

https://github.com/apache/nifi/pull/267


> Simple Username/Password Authentication
> ---
>
> Key: NIFI-1614
> URL: https://issues.apache.org/jira/browse/NIFI-1614
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: James Wing
>Assignee: James Wing
>Priority: Minor
>
> NiFi should include a simple option for username/password authentication 
> backed by a local file store.  NiFi's existing certificate and LDAP 
> authentication schemes are very secure.  However, the configuration and setup 
> is complex, making them more suitable for long-lived corporate and government 
> installations, but less accessible for casual or short-term use.  Simple 
> username/password authentication would help more users secure more NiFi 
> installations beyond anonymous admin access.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1614) Simple Username/Password Authentication

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556326#comment-15556326
 ] 

ASF GitHub Bot commented on NIFI-1614:
--

Github user jvwing commented on the issue:

https://github.com/apache/nifi/pull/267
  
Sorry, I have not kept this up to date with changes in master.  I will 
close this PR for now.  It may be a better fit for a future extension 
repository.


> Simple Username/Password Authentication
> ---
>
> Key: NIFI-1614
> URL: https://issues.apache.org/jira/browse/NIFI-1614
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: James Wing
>Assignee: James Wing
>Priority: Minor
>
> NiFi should include a simple option for username/password authentication 
> backed by a local file store.  NiFi's existing certificate and LDAP 
> authentication schemes are very secure.  However, the configuration and setup 
> is complex, making them more suitable for long-lived corporate and government 
> installations, but less accessible for casual or short-term use.  Simple 
> username/password authentication would help more users secure more NiFi 
> installations beyond anonymous admin access.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #267: NIFI-1614 File Identity Provider implementation

[jvwing]

Github user jvwing commented on the issue:

https://github.com/apache/nifi/pull/267
  
Sorry, I have not kept this up to date with changes in master.  I will 
close this PR for now.  It may be a better fit for a future extension 
repository.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #267: NIFI-1614 File Identity Provider implementation

[jvwing]

Github user jvwing closed the pull request at:

https://github.com/apache/nifi/pull/267


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-1342) PostHTTP User Agent property should be pre-populated with client default

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-1342:

Fix Version/s: (was: 0.8.0)
   0.7.1

> PostHTTP User Agent property should be pre-populated with client default
> 
>
> Key: NIFI-1342
> URL: https://issues.apache.org/jira/browse/NIFI-1342
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 0.4.1
>Reporter: Aldrin Piri
>Assignee: Pierre Villard
>Priority: Trivial
> Fix For: 1.1.0, 0.7.1
>
>
> Currently, PostHTTP shows an empty string for the User Agent property which 
> is used in web requests, but this actually results in the default of the 
> client being used.  For clarity, and if the backing library supports it, 
> getting the User Agent string used by the library as the default processor 
> User Agent property would be a nice improvement.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-1912) PutEmail doesn't send formatted email when there is an attachment with the message

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-1912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-1912:

Fix Version/s: (was: 0.8.0)
   0.7.1

> PutEmail doesn't send formatted email when there is an attachment with the 
> message
> --
>
> Key: NIFI-1912
> URL: https://issues.apache.org/jira/browse/NIFI-1912
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 0.6.1
>Reporter: Ravi Bhushan Ratnakar
>Assignee: Pierre Villard
>Priority: Critical
> Fix For: 1.1.0, 0.7.1
>
>
> When I am trying to send formatted email with an attachment using PutEmail 
> processor, all the html tags are displayed as it is like a text(This).
> After going through the code what i observed that when there is an 
> attachment, in that case content type is hard coded to 'text/plain'. Below 
> code  snippet from PutEmail 
> if (context.getProperty(ATTACH_FILE).asBoolean()) {
> final MimeBodyPart mimeText = new 
> PreencodedMimeBodyPart("base64");
> mimeText.setDataHandler(new DataHandler(new 
> ByteArrayDataSource(
> Base64.encodeBase64(messageText.getBytes("UTF-8")), 
> "text/plain; charset=\"utf-8\"")));



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2266) GetHTTP and PutHTTP use hard-coded TLS protocol version

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2266:

Fix Version/s: (was: 0.8.0)
   0.7.1

> GetHTTP and PutHTTP use hard-coded TLS protocol version
> ---
>
> Key: NIFI-2266
> URL: https://issues.apache.org/jira/browse/NIFI-2266
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 0.7.0, 0.6.1
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>  Labels: https, security, tls
> Fix For: 1.1.0, 0.7.1
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> As pointed out on the mailing list [1], the {{GetHTTP}} (and likely 
> {{PutHTTP}}) processors use a hard-coded TLS protocol version. {{PostHTTP}} 
> also did this and was fixed by [NIFI-1688]. 
> The same fix should apply here and unit tests already exist which can be 
> applied to the other processors as well. 
> For future notice, {{InvokeHTTP}} is a better processor for generic HTTP 
> operations and has supported reading the TLS protocol version from the 
> {{SSLContextService}} for some time. 
> [1] 
> https://lists.apache.org/thread.html/a48e2ebbc2231d685491ae6b856c760620efca5bff2c7249f915b24d@%3Cdev.nifi.apache.org%3E



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2280) Annotate methods as deprecated on 0.x which aren't needed on 1.x

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2280:

Fix Version/s: (was: 0.8.0)
   0.7.1

> Annotate methods as deprecated on 0.x which aren't needed on 1.x
> 
>
> Key: NIFI-2280
> URL: https://issues.apache.org/jira/browse/NIFI-2280
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joseph Witt
>Assignee: Joseph Witt
> Fix For: 0.7.1
>
>
> During the deprecated method removal/cleanup on 1.0 a few methods became no 
> longer necessary that weren't deprecated on 0.x.
> So this ticket is to mark them as deprecated on 0.x.
> They are:
> - Encrypt Process interface method for getCipher can be Deprecated. Was 
> deprecated in a couple impls.  Discussion on NIFI-1157
> - AbstractControllerService 'onConfigurationChange' is no longer needed.  Was 
> an empty impl and invoked by annotation.  Deprecate.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2298) Add missing futures for ConsumeKafka

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2298:

Fix Version/s: (was: 0.8.0)
   0.7.1

> Add missing futures for ConsumeKafka
> 
>
> Key: NIFI-2298
> URL: https://issues.apache.org/jira/browse/NIFI-2298
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 0.7.0
>Reporter: sumanth chinthagunta
>Assignee: Oleg Zhurakousky
>  Labels: kafka
> Fix For: 1.0.0, 0.7.1, 1.0.0-Beta
>
>
> The new ConsumeKafka processor  is missing some capabilities that were 
> present in old getKafka processor. 
> 1. New ConsumeKafka is not writing critical Kafka attributes  i.e., 
> kafka.key, kafka.offset, kafka.partition etc into flowFile attributes. 
> Old getKafka processor: 
> {quote}
> Standard FlowFile Attributes
> Key: 'entryDate'
>Value: 'Sun Jul 17 15:17:00 CDT 2016'
> Key: 'lineageStartDate'
>Value: 'Sun Jul 17 15:17:00 CDT 2016'
> Key: 'fileSize'
>Value: '183'
> FlowFile Attribute Map Content
> Key: 'filename'
>Value: '19709945781167274'
> Key: 'kafka.key'
>Value: '\{"database":"test","table":"sc_job","pk.systemid":1\}'
> Key: 'kafka.offset'
>Value: '1184010261'
> Key: 'kafka.partition'
>Value: '0'
> Key: 'kafka.topic'
>Value: ‘data'
> Key: 'path'
>Value: './'
> Key: 'uuid'
>Value: '244059bb-9ad9-4d74-b1fb-312eee72124a'
>  {quote}
>  
> New ConsumeKafka processor : 
>  {quote}
> Standard FlowFile Attributes
> Key: 'entryDate'
>Value: 'Sun Jul 17 15:18:41 CDT 2016'
> Key: 'lineageStartDate'
>Value: 'Sun Jul 17 15:18:41 CDT 2016'
> Key: 'fileSize'
>Value: '183'
> FlowFile Attribute Map Content
> Key: 'filename'
>Value: '19710046870478139'
> Key: 'path'
>Value: './'
> Key: 'uuid'
>Value: '349fbeb3-e342-4533-be4c-424793fa5c59’
> {quote}
> 2. getKafka/petKafka are compatible with Kafka 0.8.x and 0.9.x . 
> Please make new PublishKafka/ConsumeKafka processors based on Kafka 0.10 
> version. 
> 3. Support subscribing to multiple topics i.e., topic:  topic1,topic2 
> 4. Support configurable Serializer/DeSerializer for String, JSON , Avro etc. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2310) Shiftr Transform in JoltTransformJSON Processor Does Not Support Escaping Special Characters

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2310:

Fix Version/s: (was: 0.8.0)
   0.7.1

> Shiftr Transform in JoltTransformJSON Processor Does Not Support Escaping 
> Special Characters
> 
>
> Key: NIFI-2310
> URL: https://issues.apache.org/jira/browse/NIFI-2310
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.0.0
>Reporter: Yolanda M. Davis
>Assignee: Yolanda M. Davis
> Fix For: 1.0.0, 0.7.1, 1.0.0-Beta
>
>
> The following jolt spec failed to pass the validation check in 
> JoltTransformJSON:
> {
>   "\\@context": {
> "name": "&1.Name",
> "ingredient": "&1.Inputs",
> "yield": "\\@context.Makes",
> "*": "&1.&"
>   },
>   "name": "Name",
>   "ingredient": "Inputs",
>   "yield": "Makes",
>   "*": "&"
> }
> The reason is the double backslash to escape '@' which should be supported to 
> identify literal characters.
> Upgrading to Jolt version 0.0.21 should resolve this problem since it was 
> fixed in that release.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2322) ConsumeKafka gets stuck (cannot be stopped)

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2322:

Fix Version/s: (was: 0.8.0)
   0.7.1

> ConsumeKafka gets stuck (cannot be stopped)
> ---
>
> Key: NIFI-2322
> URL: https://issues.apache.org/jira/browse/NIFI-2322
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.0.0
>Reporter: Haimo Liu
>Assignee: Oleg Zhurakousky
> Fix For: 1.0.0, 0.7.1, 1.0.0-Beta
>
> Attachments: Screen Shot 2016-07-19 at 2.14.32 PM.png, Screen Shot 
> 2016-07-19 at 2.14.48 PM.png
>
>
> If kafka broker path is invalid or inaccurate, ConsumeKafka processor can get 
> stuck (concurrent tasks cannot be stopped in a clustered mode). Please refer 
> to the images in the attachment. It appears that a configurable timeout 
> property would potentially solve the problem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2344) non-required Controller services disabled on restart

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2344:

Fix Version/s: (was: 0.8.0)
   0.7.1

> non-required Controller services disabled on restart
> 
>
> Key: NIFI-2344
> URL: https://issues.apache.org/jira/browse/NIFI-2344
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 0.7.0
>Reporter: Brandon DeVries
>Assignee: Oleg Zhurakousky
>Priority: Blocker
> Fix For: 1.0.0, 0.7.1, 1.0.0-Beta
>
>
> So... the problem from NIFI-2160 isn't quite dead yet.  The fix for that 
> ticket recursively enumerates controller services that a controller service 
> references, but only if they are *required*\[1\].  However, there are many 
> cases in which referenced controller services may not be required properties:
> [DistributedMapCacheClientService|http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService/index.html]
>  - SSL Context Service
> [DistributedMapCacheServer|http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.distributed.cache.server.map.DistributedMapCacheServer/index.html]
>  - SSL Context Service
> [DistributedSetCacheClientService|http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.distributed.cache.client.DistributedSetCacheClientService/index.html]
>  - SSL Context Service
> [DistributedSetCacheServer|http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.distributed.cache.server.DistributedSetCacheServer/index.html]
>  - SSL Context Service
> [JMSConnectionFactoryProvider|http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.jms.cf.JMSConnectionFactoryProvider/index.html]
>  - SSL Context Service
> This causes the above services to become disabled on restart if they were 
> previously enabled.  I'd propose changing the line:
> {code}
> if (descriptor.getControllerServiceDefinition() != null && 
> descriptor.isRequired()) {
> {code}
> to:
> {code}
> if (descriptor.getControllerServiceDefinition() != null && pEntry.getValue() 
> != null) {
> {code}
> \[1\] 
> https://github.com/apache/nifi/blob/rel/nifi-0.7.0/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/service/StandardControllerServiceNode.java#L121



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2373) GetHTTP - not working with sites that only support TLS

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2373:

Fix Version/s: (was: 0.8.0)
   0.7.1

> GetHTTP - not working with sites that only support TLS
> --
>
> Key: NIFI-2373
> URL: https://issues.apache.org/jira/browse/NIFI-2373
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 0.5.1, 0.7.0
> Environment: Windows & Debian
>Reporter: Tim
>Assignee: Andy LoPresto
> Fix For: 1.1.0, 0.7.1
>
> Attachments: Test_GetHTTP.xml
>
>
> Hi,
> I'm using GetHTTP processor to download a file from nvd.nist.gov.
> Recently, they have disabled support for SSL and only allow connections to be 
> established through TLS 1.1 or TLS 1.2. 
> Since this change, the GetHTTP processor fails to download the file and 
> returns a generic "javax.net.ssl.SSLHandshakeException: Remote host closed 
> connection during handshake" error.
> How to reproduce:
>  - Create GetHTTP Processor
>  - Create StandardSSLContextService
>  - Configure URL & test download:
>https://nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-Modified.xml.gz
> What I've tried so far:
>  - Changing the SSL Protocol attribute of the SSLContextService (to TLS, TLS 
> 1.1 or TLS 1.2, no difference)
>  - Disabling SSL with the following startup argument: 
> -Dhttps.protocols=TLSv1.2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2383) ListFile does not detect new files

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2383:

Fix Version/s: (was: 0.7.1)

> ListFile does not detect new files
> --
>
> Key: NIFI-2383
> URL: https://issues.apache.org/jira/browse/NIFI-2383
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 0.7.0
>Reporter: n h
>  Labels: ListFile
>
> ListFile does not detect directory updates correctly. It seems it only check 
> for file name and modified date. 
> Using a 2 processors flow: ListFile -> FetchFile, and configure FetchFile to 
> finally delete files. Following scenario fails:
> 1. Create file1.txt somewhere outside of watching directory.
> 2. Copy file1.txt to watching directory.
> 3. Wait for FetchFile to fetch and delete file1.txt
> 4. Copy file1.txt again to watching directory.
> In the 2nd time ListFile does not detect "new copy of the same file 
> file1.txt" because file name and modified time are the same. Only creation 
> time is updated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2383) ListFile does not detect new files

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2383:

Affects Version/s: 0.7.0

> ListFile does not detect new files
> --
>
> Key: NIFI-2383
> URL: https://issues.apache.org/jira/browse/NIFI-2383
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 0.7.0
>Reporter: n h
>  Labels: ListFile
>
> ListFile does not detect directory updates correctly. It seems it only check 
> for file name and modified date. 
> Using a 2 processors flow: ListFile -> FetchFile, and configure FetchFile to 
> finally delete files. Following scenario fails:
> 1. Create file1.txt somewhere outside of watching directory.
> 2. Copy file1.txt to watching directory.
> 3. Wait for FetchFile to fetch and delete file1.txt
> 4. Copy file1.txt again to watching directory.
> In the 2nd time ListFile does not detect "new copy of the same file 
> file1.txt" because file name and modified time are the same. Only creation 
> time is updated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1112: NIFI-2799: AWS Credentials for Assume Role Need Pro...

[ktseytlin]

Github user ktseytlin commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1112#discussion_r82453054
  
--- Diff: 
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/credentials/provider/factory/strategies/AssumeRoleCredentialsStrategy.java
 ---
@@ -113,16 +134,34 @@ public AWSCredentialsProvider 
getDerivedCredentialsProvider(Map

[jira] [Commented] (NIFI-2799) AWS Credentials for Assume Role Need Proxy

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556012#comment-15556012
 ] 

ASF GitHub Bot commented on NIFI-2799:
--

Github user ktseytlin commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1112#discussion_r82453054
  
--- Diff: 
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/credentials/provider/factory/strategies/AssumeRoleCredentialsStrategy.java
 ---
@@ -113,16 +134,34 @@ public AWSCredentialsProvider 
getDerivedCredentialsProvider(Map AWS Credentials for Assume Role Need Proxy
> --
>
> Key: NIFI-2799
> URL: https://issues.apache.org/jira/browse/NIFI-2799
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Keren Tseytlin
>Assignee: James Wing
>Priority: Minor
> Fix For: 1.1.0
>
>
> As a user of Nifi, when I want to enable cross account fetching of S3 objects 
> I need the proxy variables to be set in order to generate temporary AWS 
> tokens for STS:AssumeRole.
> Within some enterprise environments, it is necessary to set the proxy 
> variables prior to running AssumeRole methods. Without this being set, the 
> machine in VPC A times out on generating temporary keys and is unable to 
> assume a role as a machine in VPC B. 
> This ticket arose from this conversation: 
> http://apache-nifi-developer-list.39713.n7.nabble.com/Nifi-Cross-Account-Download-With-A-Profile-Flag-td13232.html#a13252
> Goal: There are files stored in an S3 bucket in VPC B. My Nifi machines are 
> in VPC A. I want Nifi to be able to get those files from VPC B. VPC A and VPC 
> B need to be communicating in the FetchS3Object component.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2411) ModifyBytes should use long instead of int for offsets.

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2411:

Fix Version/s: (was: 0.8.0)
   0.7.1

> ModifyBytes should use long instead of int for offsets.
> ---
>
> Key: NIFI-2411
> URL: https://issues.apache.org/jira/browse/NIFI-2411
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Joe Skora
>Assignee: Joe Skora
>  Labels: easyfix
> Fix For: 1.0.0, 0.7.1
>
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> ModifyBytes.onTrigger() uses Java 32 bit {{int}} value for byte offsets 
> limiting it to 2 Gigabytes, switching to {{long}} values will allow it to 
> handle up to 15 Exabytes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2412) Allow user to set serializer and de serializer in publish and consume kafka processors

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2412:

Fix Version/s: (was: 0.8.0)
   0.7.1

> Allow user to set serializer and de serializer in publish and consume kafka 
> processors
> --
>
> Key: NIFI-2412
> URL: https://issues.apache.org/jira/browse/NIFI-2412
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 0.7.0
>Reporter: Arpit Gupta
>Assignee: Oleg Zhurakousky
> Fix For: 1.0.0, 0.7.1, 1.0.0-Beta
>
>
> Publish kafka today sets key and value serializer to class 
> org.apache.kafka.common.serialization.ByteArraySerializer
> Even if the user passes different values in the processor config they get 
> overwritten.
> We should allow dynamic values for serializer and de serializer as needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2423) Add a sensitive property to store the trust store password in the consume/publish kafka processors

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2423:

Fix Version/s: (was: 0.8.0)
   0.7.1

> Add a sensitive property to store the trust store password in the 
> consume/publish kafka processors
> --
>
> Key: NIFI-2423
> URL: https://issues.apache.org/jira/browse/NIFI-2423
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 0.7.0
>Reporter: Arpit Gupta
>Assignee: Oleg Zhurakousky
>Priority: Critical
> Fix For: 1.0.0, 0.7.1
>
> Attachments: 
> 0001-NIFI-2423-Make-use-of-the-SSLContextService-to-provi.patch
>
>
> When user configures kafka processors to interact with SSL enabled protocol 
> they need to set additional configs.
> ssl.truststore.location and ssl.truststore.password. 
> Today the trust store password is clear text. We should make that a default 
> property and make it sensitive.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2799) AWS Credentials for Assume Role Need Proxy

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1991#comment-1991
 ] 

ASF GitHub Bot commented on NIFI-2799:
--

Github user ktseytlin commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1112#discussion_r82451576
  
--- Diff: 
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/credentials/provider/factory/strategies/AssumeRoleCredentialsStrategy.java
 ---
@@ -113,16 +134,34 @@ public AWSCredentialsProvider 
getDerivedCredentialsProvider(Map(STSAssumeRoleSessionCredentialsProvider.java:187)
at 
com.amazonaws.auth.STSAssumeRoleSessionCredentialsProvider.(STSAssumeRoleSessionCredentialsProvider.java:34)
at 
com.amazonaws.auth.STSAssumeRoleSessionCredentialsProvider$Builder.build(STSAssumeRoleSessionCredentialsProvider.java:436)
```

I wanted to refactor it like that, but again, since I am not able to test 
in my environment the previous work I played it on the safe side.


> AWS Credentials for Assume Role Need Proxy
> --
>
> Key: NIFI-2799
> URL: https://issues.apache.org/jira/browse/NIFI-2799
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Keren Tseytlin
>Assignee: James Wing
>Priority: Minor
> Fix For: 1.1.0
>
>
> As a user of Nifi, when I want to enable cross account fetching of S3 objects 
> I need the proxy variables to be set in order to generate temporary AWS 
> tokens for STS:AssumeRole.
> Within some enterprise environments, it is necessary to set the proxy 
> variables prior to running AssumeRole methods. Without this being set, the 
> machine in VPC A times out on generating temporary keys and is unable to 
> assume a role as a machine in VPC B. 
> This ticket arose from this conversation: 
> http://apache-nifi-developer-list.39713.n7.nabble.com/Nifi-Cross-Account-Download-With-A-Profile-Flag-td13232.html#a13252
> Goal: There are files stored in an S3 bucket in VPC B. My Nifi machines are 
> in VPC A. I want Nifi to be able to get those files from VPC B. VPC A and VPC 
> B need to be communicating in the FetchS3Object component.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1112: NIFI-2799: AWS Credentials for Assume Role Need Pro...

[ktseytlin]

Github user ktseytlin commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1112#discussion_r82451576
  
--- Diff: 
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/credentials/provider/factory/strategies/AssumeRoleCredentialsStrategy.java
 ---
@@ -113,16 +134,34 @@ public AWSCredentialsProvider 
getDerivedCredentialsProvider(Map(STSAssumeRoleSessionCredentialsProvider.java:187)
at 
com.amazonaws.auth.STSAssumeRoleSessionCredentialsProvider.(STSAssumeRoleSessionCredentialsProvider.java:34)
at 
com.amazonaws.auth.STSAssumeRoleSessionCredentialsProvider$Builder.build(STSAssumeRoleSessionCredentialsProvider.java:436)
```

I wanted to refactor it like that, but again, since I am not able to test 
in my environment the previous work I played it on the safe side.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-2429) Persistent provenance repo should continue despite indexing failures 0.x branch improvement

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2429:

   Resolution: Fixed
Fix Version/s: (was: 0.8.0)
   0.7.1
   Status: Resolved  (was: Patch Available)

> Persistent provenance repo should continue despite indexing failures 0.x 
> branch improvement
> ---
>
> Key: NIFI-2429
> URL: https://issues.apache.org/jira/browse/NIFI-2429
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core Framework
>Affects Versions: 0.7.0
>Reporter: Joseph Witt
>Assignee: Mark Payne
>Priority: Critical
> Fix For: 0.7.1
>
>
> [~markap14] created this subtask for just the 0.x line so we can get the 1.x 
> ticket addressed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2799) AWS Credentials for Assume Role Need Proxy

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1979#comment-1979
 ] 

ASF GitHub Bot commented on NIFI-2799:
--

Github user jvwing commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1112#discussion_r82451044
  
--- Diff: 
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/credentials/provider/factory/strategies/AssumeRoleCredentialsStrategy.java
 ---
@@ -113,16 +134,34 @@ public AWSCredentialsProvider 
getDerivedCredentialsProvider(Map AWS Credentials for Assume Role Need Proxy
> --
>
> Key: NIFI-2799
> URL: https://issues.apache.org/jira/browse/NIFI-2799
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Keren Tseytlin
>Assignee: James Wing
>Priority: Minor
> Fix For: 1.1.0
>
>
> As a user of Nifi, when I want to enable cross account fetching of S3 objects 
> I need the proxy variables to be set in order to generate temporary AWS 
> tokens for STS:AssumeRole.
> Within some enterprise environments, it is necessary to set the proxy 
> variables prior to running AssumeRole methods. Without this being set, the 
> machine in VPC A times out on generating temporary keys and is unable to 
> assume a role as a machine in VPC B. 
> This ticket arose from this conversation: 
> http://apache-nifi-developer-list.39713.n7.nabble.com/Nifi-Cross-Account-Download-With-A-Profile-Flag-td13232.html#a13252
> Goal: There are files stored in an S3 bucket in VPC B. My Nifi machines are 
> in VPC A. I want Nifi to be able to get those files from VPC B. VPC A and VPC 
> B need to be communicating in the FetchS3Object component.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2445) PublishKafka sends flowfiles to 'success' if not authorized to write to topic

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2445:

Fix Version/s: (was: 0.8.0)
   0.7.1

> PublishKafka sends flowfiles to 'success' if not authorized to write to topic
> -
>
> Key: NIFI-2445
> URL: https://issues.apache.org/jira/browse/NIFI-2445
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Oleg Zhurakousky
>Priority: Blocker
> Fix For: 1.0.0, 0.7.1, 1.0.0-Beta
>
>
> I have a PublishKafka processor configured to write to a kafka topic that I 
> am not authorized to write to (via PLAINTEXT_SASL configuration). When I 
> attempt to send a message to Kafka, I get a stack trace telling me that I am 
> not authorized but the FlowFile is still routed to 'success'



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2471) Multiple Hadoop processors cannot point to different Hadoop clusters

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2471:

Fix Version/s: (was: 0.8.0)
   0.7.1

> Multiple Hadoop processors cannot point to different Hadoop clusters
> 
>
> Key: NIFI-2471
> URL: https://issues.apache.org/jira/browse/NIFI-2471
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 0.7.0
>Reporter: Michael Moser
>Assignee: Michael Moser
>Priority: Minor
> Fix For: 1.0.0, 0.7.1, 1.0.0-Beta
>
>
> Two GetHDFS processors, each with different Hadoop Configuration Resources, 
> will only operate with one Hadoop cluster.  In this specific case, both 
> Hadoop clusters had the same HDFS system name.  The AbstractHadoopProcessor 
> disables caching of Configuration and FileSystem objects but it doesn't 
> appear to be working.
> Also, if I configure a GetHDFS processor to point to one Hadoop cluster, but 
> the processor is invalid because the Directory doesn't exist, the processor 
> doesn't start (that's good).  But if I change the Hadoop Configuration 
> Resources to point to a different Hadoop cluster where the Directory does 
> exist, then GetHDFS continues to talk to the first Hadoop cluster.  It seems 
> like the Configuration and FileSystem objects don't reset when I change the 
> Hadoop Configuration Resources property.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2503) Backport PostHTTP SSL Protocol fix to 0.x branch

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2503:

Fix Version/s: (was: 0.8.0)
   0.7.1

> Backport PostHTTP SSL Protocol fix to 0.x branch
> 
>
> Key: NIFI-2503
> URL: https://issues.apache.org/jira/browse/NIFI-2503
> Project: Apache NiFi
>  Issue Type: Sub-task
>  Components: Core Framework
>Affects Versions: 0.7.0
>Reporter: Joe Skora
>Assignee: Joe Skora
>Priority: Critical
> Fix For: 0.7.1
>
>
> PostHTTP in the 0.x branch still has the hardcoded protocol version that 
> NIFI-1688 / [PR-624|https://github.com/apache/nifi/pull/624] fixed in the 1.0 
> branch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1112: NIFI-2799: AWS Credentials for Assume Role Need Pro...

[jvwing]

Github user jvwing commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1112#discussion_r82451044
  
--- Diff: 
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/credentials/provider/factory/strategies/AssumeRoleCredentialsStrategy.java
 ---
@@ -113,16 +134,34 @@ public AWSCredentialsProvider 
getDerivedCredentialsProvider(Map

[jira] [Updated] (NIFI-2508) Revert to using getKeyStorePassword() for key-password in AbstractKafkaProcessor

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2508:

Fix Version/s: (was: 0.8.0)
   0.7.1

> Revert to using getKeyStorePassword() for key-password in 
> AbstractKafkaProcessor
> 
>
> Key: NIFI-2508
> URL: https://issues.apache.org/jira/browse/NIFI-2508
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Oleg Zhurakousky
>Assignee: Oleg Zhurakousky
>Priority: Blocker
> Fix For: 0.7.1
>
>
> Current NIFI build for 0.x is broken due to the SSL changes in Kafka 
> processors where it is using an _getKeyPassword()_ operation only available 
> in 1.0. We need to fall back on _getKeyStorePassword()_.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2509) KafkaConsumer results in NPE if check.connection property is not provided

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2509:

Fix Version/s: (was: 0.8.0)
   0.7.1

> KafkaConsumer results in NPE if check.connection property is not provided
> -
>
> Key: NIFI-2509
> URL: https://issues.apache.org/jira/browse/NIFI-2509
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.0.0-Beta
>Reporter: Arpit Gupta
>Assignee: Oleg Zhurakousky
> Fix For: 1.0.0, 0.7.1
>
>
> If the user does not provide check connection property consume kafka returns 
> a NPE
> {code}
> java.lang.NullPointerException: null
> at 
> org.apache.nifi.processors.kafka.pubsub.ConsumeKafka.buildKafkaResource(ConsumeKafka.java:241)
>  
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2521) License and Notice - TestSNMPAgent classes appear to be copied from external sources

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2521:

Fix Version/s: (was: 0.8.0)
   0.7.1

> License and Notice - TestSNMPAgent classes appear to be copied from external 
> sources
> 
>
> Key: NIFI-2521
> URL: https://issues.apache.org/jira/browse/NIFI-2521
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joseph Witt
>Assignee: Joseph Witt
>Priority: Blocker
> Fix For: 1.0.0, 0.7.1
>
>
> TestSNMPAgentV1 and TestSNMPAgentV2c appear to be sourced at least in large 
> part from https://code.google.com/archive/p/springside-sub/
> TestSNMPAgentV3 appears to be sourced from  XYZ as found here 
> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-snmp-bundle/nifi-snmp-processors/src/test/java/org/apache/nifi/snmp/processors/TestSnmpAgentV3.java#L68.
> These classes should be completely removed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2515) Setting key.serializer in publish kafka leads to org.apache.kafka.common.errors.SerializationException

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2515:

Fix Version/s: (was: 0.8.0)
   0.7.1

> Setting key.serializer in publish kafka leads to 
> org.apache.kafka.common.errors.SerializationException
> --
>
> Key: NIFI-2515
> URL: https://issues.apache.org/jira/browse/NIFI-2515
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.0.0-Beta
>Reporter: Arpit Gupta
>Assignee: Oleg Zhurakousky
>Priority: Minor
> Fix For: 1.0.0, 0.7.1
>
>
> If one sets key.serializer in the publish kafka processor we see error
> {code}
> 4895-9b14-9e8b1d88755e}, body={topics=[putMessages-83cf2948-b110-2016-08-07 
> 17:11:56,191 ERROR [Timer-Driven Process Thread-7] 
> o.a.n.p.kafka.pubsub.PublishKafka 
> PublishKafka[id=65fd2baf-0156-1000--f6bf89be] 
> PublishKafka[id=65fd2baf-0156-1000--f6bf89be] failed to process due 
> to org.apache.kafka.common.errors.SerializationException: Can't convert key 
> of class [B to class org.apache.kafka.common.serialization.IntegerSerializer 
> specified in key.serializer; rolling back session: 
> org.apache.kafka.common.errors.SerializationException: Can't convert key of 
> class [B to class org.apache.kafka.common.serialization.IntegerSerializer 
> specified in key.serializer
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2551) Rare condition causes FileSystemRepository NPE

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2551:

Fix Version/s: (was: 0.8.0)
   0.7.1

> Rare condition causes FileSystemRepository NPE
> --
>
> Key: NIFI-2551
> URL: https://issues.apache.org/jira/browse/NIFI-2551
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Michael Moser
>Assignee: Mark Payne
>Priority: Blocker
> Fix For: 1.0.0, 0.7.1
>
>
> In rare unpredictable cases when NiFi is processing a heavy load, we see 
> FileSystemRepository throw a NullPointerException
> {noformat}
> java.lang.NullPointerException
> at o.a.n.c.r.FileSystemRepository$2.write(FileSystemRepository.java:918) 
> [nifi-framework-core-0.7.0.jar]
> at 
> o.a.n.c.r.io.DisableOnCloseOutputStream.write(DisableOnCloseOutputStream.java:49)
> 
> Suppressed: java.lang.NullPointerException
> at 
> o.a.n.c.r.FileSystemRepository$2.flush(FileSystemRepository.java:934) 
> [nifi-framework-core-0.7.0.jar]
> at 
> o.a.n.c.r.io.DisableOnCloseOutputStream.close(DisableOnCloseOutputStream.java:68)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2611) UnpackContent cannot unpack any type of flowfile

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2611:

Fix Version/s: (was: 0.8.0)
   0.7.1

> UnpackContent cannot unpack any type of flowfile
> 
>
> Key: NIFI-2611
> URL: https://issues.apache.org/jira/browse/NIFI-2611
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 0.7.0
>Reporter: Joseph Gresock
>Assignee: Joseph Gresock
> Fix For: 1.0.0, 0.7.1
>
>
> Two possibly separate problems:
> *Flowfile-stream-v2 and v3*
> This may be a problem with either MergeContent's production of 
> flowfile-stream v2 and v3, or with UnpackContent's inability to unpack them, 
> not sure which.  Here is a screen shot with how to reproduce it: 
> https://ibin.co/2sCwqbFbAs3a.png
> Essentially, when you pack a flow file as flowfile-stream v2 or v3, a 
> subsequent UnpackContent set to the respective type fails with the error 
> "Cannot unpack {} because it does not appear to have any entries".
> *Flowfile-tar-v1*
> When selecting flowfile-tar-v1 from UnpackContent, you immediately get 
> @OnScheduled error failure as soon as you start the processor, which prevents 
> it from processing any incoming flow files.  Here is a screenshot: 
> https://ibin.co/2sCxI4iDm88t.png



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2799) AWS Credentials for Assume Role Need Proxy

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1967#comment-1967
 ] 

ASF GitHub Bot commented on NIFI-2799:
--

Github user ktseytlin commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1112#discussion_r82450516
  
--- Diff: 
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/credentials/provider/factory/strategies/AssumeRoleCredentialsStrategy.java
 ---
@@ -113,16 +134,34 @@ public AWSCredentialsProvider 
getDerivedCredentialsProvider(Map AWS Credentials for Assume Role Need Proxy
> --
>
> Key: NIFI-2799
> URL: https://issues.apache.org/jira/browse/NIFI-2799
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Keren Tseytlin
>Assignee: James Wing
>Priority: Minor
> Fix For: 1.1.0
>
>
> As a user of Nifi, when I want to enable cross account fetching of S3 objects 
> I need the proxy variables to be set in order to generate temporary AWS 
> tokens for STS:AssumeRole.
> Within some enterprise environments, it is necessary to set the proxy 
> variables prior to running AssumeRole methods. Without this being set, the 
> machine in VPC A times out on generating temporary keys and is unable to 
> assume a role as a machine in VPC B. 
> This ticket arose from this conversation: 
> http://apache-nifi-developer-list.39713.n7.nabble.com/Nifi-Cross-Account-Download-With-A-Profile-Flag-td13232.html#a13252
> Goal: There are files stored in an S3 bucket in VPC B. My Nifi machines are 
> in VPC A. I want Nifi to be able to get those files from VPC B. VPC A and VPC 
> B need to be communicating in the FetchS3Object component.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2618) TestPostHTTPGroovy test in 0.x branch fails using Java 7

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2618:

Fix Version/s: (was: 0.8.0)
   0.7.1

> TestPostHTTPGroovy test in 0.x branch fails using Java 7
> 
>
> Key: NIFI-2618
> URL: https://issues.apache.org/jira/browse/NIFI-2618
> Project: Apache NiFi
>  Issue Type: Task
>  Components: Core Framework
>Affects Versions: 0.7.0
>Reporter: Michael Moser
>Assignee: Andy LoPresto
>Priority: Trivial
> Fix For: 0.7.1
>
>
> The TestPostHTTPGroovy unit test testDefaultShouldPreferTLSv1_2 fails on the 
> 0.x branch when using Java 7 to build.  The default TLS in Java 7 is TLSv1.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2631) ListS3 improvements: "Use versions" and "Commit mode"

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2631:

Fix Version/s: (was: 0.8.0)
   0.7.1

> ListS3 improvements: "Use versions" and "Commit mode"
> -
>
> Key: NIFI-2631
> URL: https://issues.apache.org/jira/browse/NIFI-2631
> Project: Apache NiFi
>  Issue Type: Improvement
>Affects Versions: 0.7.0
>Reporter: Joseph Gresock
>Assignee: Joseph Gresock
>Priority: Minor
> Fix For: 1.1.0, 0.7.1
>
>
> Our team needs to be able to list individual versions in S3.  We also ran 
> into a use case where a bucket with many objects (over 1 million in our case) 
> seemed to cause ListS3 to run forever.  The S3 list command finished in a few 
> minutes, but we believe it was taking a very long time for NiFi to commit all 
> the flow files at once.
> To handle this use case, we added a Commit Mode property to ListS3 that 
> allows you specify that you want to commit "Per page" vs. "Once".  This has 
> proven to correctly emit the flow files as the S3 paging progresses.
> We also implemented support for S3 List Versions, which includes the 
> "s3.version" and "s3.isLatest" attributes if applicable.  The "s3.version" 
> attribute can in turn be used in the FetchS3 processor.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1112: NIFI-2799: AWS Credentials for Assume Role Need Pro...

[ktseytlin]

Github user ktseytlin commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1112#discussion_r82450516
  
--- Diff: 
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/credentials/provider/factory/strategies/AssumeRoleCredentialsStrategy.java
 ---
@@ -113,16 +134,34 @@ public AWSCredentialsProvider 
getDerivedCredentialsProvider(Map

[jira] [Updated] (NIFI-2680) PutKafka processors can fail but sill transfer the flowfile to the success output

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2680:

Fix Version/s: (was: 0.8.0)
   0.7.1

> PutKafka processors can fail but sill transfer the flowfile to the success 
> output
> -
>
> Key: NIFI-2680
> URL: https://issues.apache.org/jira/browse/NIFI-2680
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Christopher McDermott
>Priority: Critical
> Fix For: 1.0.0, 0.7.1
>
>
>  
> I have PutKafka processor where I’ve set the Max Record Size property to 1MB.
>  
> If a transfer file larger than 1MiB to it, I see a bulletin and the following 
> message in the log file, but the flowfile is transferred to the success 
> output.
> This can lead to data loss in the flow.
>  
> Caused by: org.apache.kafka.common.errors.RecordTooLargeException: The 
> message is 5800160 bytes when serialized which is larger than the maximum 
> request size you have configured with the max.request.si\
> ze configuration.
> 2016-08-25 20:38:22,041 ERROR [Timer-Driven Process Thread-7] 
> o.apache.nifi.processors.kafka.PutKafka 
> PutKafka[id=3c396e30-7294-467e-b5d7-a3d44c0a04f4] Failed while waiting for 
> acks from Kafka
> 2016-08-25 20:38:22,042 ERROR [Timer-Driven Process Thread-7] 
> o.apache.nifi.processors.kafka.PutKafka
> java.util.concurrent.ExecutionException: 
> org.apache.kafka.common.errors.RecordTooLargeException: The message is 
> 5800160 bytes when serialized which is larger than the maximum request size 
> you have conf\
> igured with the max.request.size configuration.
> at 
> org.apache.kafka.clients.producer.KafkaProducer$FutureFailure.(KafkaProducer.java:437)
>  ~[kafka-clients-0.8.2.2.jar:na]
> at 
> org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:352) 
> ~[kafka-clients-0.8.2.2.jar:na]
> at 
> org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:248) 
> ~[kafka-clients-0.8.2.2.jar:na]
> at 
> org.apache.nifi.processors.kafka.KafkaPublisher.publish(KafkaPublisher.java:137)
>  ~[nifi-kafka-processors-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at 
> org.apache.nifi.processors.kafka.PutKafka$1.process(PutKafka.java:315) 
> [nifi-kafka-processors-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at 
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1851)
>  [nifi-framework-core-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at 
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1822)
>  [nifi-framework-core-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at 
> org.apache.nifi.processors.kafka.PutKafka.doRendezvousWithKafka(PutKafka.java:311)
>  [nifi-kafka-processors-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at 
> org.apache.nifi.processors.kafka.PutKafka.rendezvousWithKafka(PutKafka.java:287)
>  [nifi-kafka-processors-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at 
> org.apache.nifi.processors.kafka.AbstractKafkaProcessor.onTrigger(AbstractKafkaProcessor.java:76)
>  [nifi-kafka-processors-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at 
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1054)
>  [nifi-framework-core-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
>  [nifi-framework-core-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
>  [nifi-framework-core-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at 
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:127)
>  [nifi-framework-core-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_45]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
> [na:1.8.0_45]
>  
> I have not yet tried the ProduceKafka processor



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2636) UnpackContent has concurrent thread safety issue, causes flowfiles to fail

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2636:

Fix Version/s: (was: 0.8.0)
   0.7.1

> UnpackContent has concurrent thread safety issue, causes flowfiles to fail
> --
>
> Key: NIFI-2636
> URL: https://issues.apache.org/jira/browse/NIFI-2636
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Michael Moser
>Assignee: Michael Moser
> Fix For: 1.1.0, 0.7.1
>
>
> Shortly after merging NIFI-2611 I took a last look at the code and noticed 
> that each onTrigger() call, when the Packaging Format property is set to "use 
> mime.type attribute", that the class instance variable "private Unpacker 
> unpacker" can change.  When UnpackContent is set to > 1 concurrent task, this 
> isn't thread safe.  Thread A can set the unpacker to the TarUnpacker, but 
> before it gets a chance to unpack its tar file, Thread B changes the unpacker 
> to a FlowFileUnpackagerV3 which causes Thread A to fail its unpack.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2680) PutKafka processors can fail but sill transfer the flowfile to the success output

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2680:

Affects Version/s: (was: 0.8.0)

> PutKafka processors can fail but sill transfer the flowfile to the success 
> output
> -
>
> Key: NIFI-2680
> URL: https://issues.apache.org/jira/browse/NIFI-2680
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Christopher McDermott
>Priority: Critical
> Fix For: 1.0.0, 0.8.0
>
>
>  
> I have PutKafka processor where I’ve set the Max Record Size property to 1MB.
>  
> If a transfer file larger than 1MiB to it, I see a bulletin and the following 
> message in the log file, but the flowfile is transferred to the success 
> output.
> This can lead to data loss in the flow.
>  
> Caused by: org.apache.kafka.common.errors.RecordTooLargeException: The 
> message is 5800160 bytes when serialized which is larger than the maximum 
> request size you have configured with the max.request.si\
> ze configuration.
> 2016-08-25 20:38:22,041 ERROR [Timer-Driven Process Thread-7] 
> o.apache.nifi.processors.kafka.PutKafka 
> PutKafka[id=3c396e30-7294-467e-b5d7-a3d44c0a04f4] Failed while waiting for 
> acks from Kafka
> 2016-08-25 20:38:22,042 ERROR [Timer-Driven Process Thread-7] 
> o.apache.nifi.processors.kafka.PutKafka
> java.util.concurrent.ExecutionException: 
> org.apache.kafka.common.errors.RecordTooLargeException: The message is 
> 5800160 bytes when serialized which is larger than the maximum request size 
> you have conf\
> igured with the max.request.size configuration.
> at 
> org.apache.kafka.clients.producer.KafkaProducer$FutureFailure.(KafkaProducer.java:437)
>  ~[kafka-clients-0.8.2.2.jar:na]
> at 
> org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:352) 
> ~[kafka-clients-0.8.2.2.jar:na]
> at 
> org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:248) 
> ~[kafka-clients-0.8.2.2.jar:na]
> at 
> org.apache.nifi.processors.kafka.KafkaPublisher.publish(KafkaPublisher.java:137)
>  ~[nifi-kafka-processors-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at 
> org.apache.nifi.processors.kafka.PutKafka$1.process(PutKafka.java:315) 
> [nifi-kafka-processors-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at 
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1851)
>  [nifi-framework-core-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at 
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1822)
>  [nifi-framework-core-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at 
> org.apache.nifi.processors.kafka.PutKafka.doRendezvousWithKafka(PutKafka.java:311)
>  [nifi-kafka-processors-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at 
> org.apache.nifi.processors.kafka.PutKafka.rendezvousWithKafka(PutKafka.java:287)
>  [nifi-kafka-processors-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at 
> org.apache.nifi.processors.kafka.AbstractKafkaProcessor.onTrigger(AbstractKafkaProcessor.java:76)
>  [nifi-kafka-processors-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at 
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1054)
>  [nifi-framework-core-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
>  [nifi-framework-core-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
>  [nifi-framework-core-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at 
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:127)
>  [nifi-framework-core-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_45]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
> [na:1.8.0_45]
>  
> I have not yet tried the ProduceKafka processor



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2789) Add JMS properties to FlowFile attributes on receive in ConsumeJMS

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2789:

Fix Version/s: (was: 0.8.0)
   0.7.1

> Add JMS properties to FlowFile attributes on receive in ConsumeJMS
> --
>
> Key: NIFI-2789
> URL: https://issues.apache.org/jira/browse/NIFI-2789
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joey Frazee
>Assignee: Joey Frazee
>Priority: Minor
> Fix For: 1.1.0, 0.7.1
>
>
> ConsumeJMS currently adds JMS headers to the FlowFile attributes when it 
> receives a message but ignores any JMS properties coming through. It should 
> be reading both the headers and properties and merging them into the FlowFile 
> attributes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2790) Set JMS destination name on send/receive instead of using the default destination

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2790:

Fix Version/s: (was: 0.8.0)
   0.7.1

> Set JMS destination name on send/receive instead of using the default 
> destination
> -
>
> Key: NIFI-2790
> URL: https://issues.apache.org/jira/browse/NIFI-2790
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joey Frazee
>Assignee: Joey Frazee
>Priority: Minor
> Fix For: 1.1.0, 0.7.1
>
>
> ConsumeJMS and PublishJMS currently pull their destination name from the 
> default JMS destination (setDefaultDestinationName() on the JmsTemplate). The 
> effect this has is that attribute expressions are evaluated with respect to 
> the context only and not the FlowFile, so expression language support really 
> only extends to EL functions and variables from the variable registry.
> This doesn't have a big impact on ConsumeJMS since it doesn't take input, but 
> it means that destinations can be set at runtime in PublishJMS.
> The JmsTemplate send() and receive() can take the destination name as an 
> argument though, so these method variants should be used so EL support is 
> fully enabled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2799) AWS Credentials for Assume Role Need Proxy

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1901#comment-1901
 ] 

ASF GitHub Bot commented on NIFI-2799:
--

Github user ktseytlin commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1112#discussion_r82446283
  
--- Diff: 
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/credentials/provider/factory/strategies/AssumeRoleCredentialsStrategy.java
 ---
@@ -113,16 +134,34 @@ public AWSCredentialsProvider 
getDerivedCredentialsProvider(Map AWS Credentials for Assume Role Need Proxy
> --
>
> Key: NIFI-2799
> URL: https://issues.apache.org/jira/browse/NIFI-2799
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Keren Tseytlin
>Assignee: James Wing
>Priority: Minor
> Fix For: 1.1.0
>
>
> As a user of Nifi, when I want to enable cross account fetching of S3 objects 
> I need the proxy variables to be set in order to generate temporary AWS 
> tokens for STS:AssumeRole.
> Within some enterprise environments, it is necessary to set the proxy 
> variables prior to running AssumeRole methods. Without this being set, the 
> machine in VPC A times out on generating temporary keys and is unable to 
> assume a role as a machine in VPC B. 
> This ticket arose from this conversation: 
> http://apache-nifi-developer-list.39713.n7.nabble.com/Nifi-Cross-Account-Download-With-A-Profile-Flag-td13232.html#a13252
> Goal: There are files stored in an S3 bucket in VPC B. My Nifi machines are 
> in VPC A. I want Nifi to be able to get those files from VPC B. VPC A and VPC 
> B need to be communicating in the FetchS3Object component.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1112: NIFI-2799: AWS Credentials for Assume Role Need Pro...

[ktseytlin]

Github user ktseytlin commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1112#discussion_r82446283
  
--- Diff: 
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/credentials/provider/factory/strategies/AssumeRoleCredentialsStrategy.java
 ---
@@ -113,16 +134,34 @@ public AWSCredentialsProvider 
getDerivedCredentialsProvider(Map

[jira] [Updated] (NIFI-2874) StreamDemarcator can return wrong data for token

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-2874:

Resolution: Fixed
Status: Resolved  (was: Patch Available)

[~markap14] would you close https://github.com/apache/nifi/pull/1114?  Thx.

> StreamDemarcator can return wrong data for token
> 
>
> Key: NIFI-2874
> URL: https://issues.apache.org/jira/browse/NIFI-2874
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Critical
> Fix For: 1.1.0, 0.7.1
>
>
> There is a case where StreamDemarcator can return the wrong data for a token. 
> If a token ends at the end of the buffer, and the next token is smaller than 
> the previous, it can result in the next token keeping part of the buffer's 
> content. The code below is a unit test that exposes this:
> {code}
> @Test
> public void testOnBufferSplitNoTrailingDelimiter() throws IOException {
> final byte[] inputData = "Yes\nNo".getBytes(StandardCharsets.UTF_8);
> ByteArrayInputStream is = new ByteArrayInputStream(inputData);
> StreamDemarcator scanner = new StreamDemarcator(is, "\n".getBytes(), 
> 1000, 3);
> final byte[] first = scanner.nextToken();
> final byte[] second = scanner.nextToken();
> assertNotNull(first);
> assertNotNull(second);
> assertArrayEquals(first, new byte[] {'Y', 'e', 's'});
> assertArrayEquals(second, new byte[] {'N', 'o'});
> }
> {code}
> In this case, the second token, which should be 'No' comes back as 'Nos' 
> because it contains the 's' from the previous token.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2874) StreamDemarcator can return wrong data for token

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1868#comment-1868
 ] 

ASF GitHub Bot commented on NIFI-2874:
--

Github user mosermw commented on the issue:

https://github.com/apache/nifi/pull/1114
  
Looked great to me, merged to 0.x.


> StreamDemarcator can return wrong data for token
> 
>
> Key: NIFI-2874
> URL: https://issues.apache.org/jira/browse/NIFI-2874
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Critical
> Fix For: 1.1.0, 0.7.1
>
>
> There is a case where StreamDemarcator can return the wrong data for a token. 
> If a token ends at the end of the buffer, and the next token is smaller than 
> the previous, it can result in the next token keeping part of the buffer's 
> content. The code below is a unit test that exposes this:
> {code}
> @Test
> public void testOnBufferSplitNoTrailingDelimiter() throws IOException {
> final byte[] inputData = "Yes\nNo".getBytes(StandardCharsets.UTF_8);
> ByteArrayInputStream is = new ByteArrayInputStream(inputData);
> StreamDemarcator scanner = new StreamDemarcator(is, "\n".getBytes(), 
> 1000, 3);
> final byte[] first = scanner.nextToken();
> final byte[] second = scanner.nextToken();
> assertNotNull(first);
> assertNotNull(second);
> assertArrayEquals(first, new byte[] {'Y', 'e', 's'});
> assertArrayEquals(second, new byte[] {'N', 'o'});
> }
> {code}
> In this case, the second token, which should be 'No' comes back as 'Nos' 
> because it contains the 's' from the previous token.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1114: NIFI-2874: Ensure that when reading more data from an Inpu...

[mosermw]

Github user mosermw commented on the issue:

https://github.com/apache/nifi/pull/1114
  
Looked great to me, merged to 0.x.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2874) StreamDemarcator can return wrong data for token

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1864#comment-1864
 ] 

ASF subversion and git services commented on NIFI-2874:
---

Commit 5c0c1d3bb41fa75622a3a7502ccf86ec49d2fe4d in nifi's branch refs/heads/0.x 
from [~markap14]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=5c0c1d3 ]

NIFI-2874: Ensure that when reading more data from an InputStream the 
StreamDemarcator appropriately updates the max index that can be read from the 
buffer

Signed-off-by: Mike Moser 

This closes #1114.


> StreamDemarcator can return wrong data for token
> 
>
> Key: NIFI-2874
> URL: https://issues.apache.org/jira/browse/NIFI-2874
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Critical
> Fix For: 1.1.0, 0.7.1
>
>
> There is a case where StreamDemarcator can return the wrong data for a token. 
> If a token ends at the end of the buffer, and the next token is smaller than 
> the previous, it can result in the next token keeping part of the buffer's 
> content. The code below is a unit test that exposes this:
> {code}
> @Test
> public void testOnBufferSplitNoTrailingDelimiter() throws IOException {
> final byte[] inputData = "Yes\nNo".getBytes(StandardCharsets.UTF_8);
> ByteArrayInputStream is = new ByteArrayInputStream(inputData);
> StreamDemarcator scanner = new StreamDemarcator(is, "\n".getBytes(), 
> 1000, 3);
> final byte[] first = scanner.nextToken();
> final byte[] second = scanner.nextToken();
> assertNotNull(first);
> assertNotNull(second);
> assertArrayEquals(first, new byte[] {'Y', 'e', 's'});
> assertArrayEquals(second, new byte[] {'N', 'o'});
> }
> {code}
> In this case, the second token, which should be 'No' comes back as 'Nos' 
> because it contains the 's' from the previous token.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1526) Allow components to provide default values for Yield Duration and Run Schedule

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1844#comment-1844
 ] 

ASF GitHub Bot commented on NIFI-1526:
--

Github user mathiastiberghien commented on the issue:

https://github.com/apache/nifi/pull/1107
  
I'll correct styles (I'm really newbie to java and open source).
I add issues to launch the styles tests so  I hope I won't miss anything.
Sorry for the undesired change. I had to put a hard code reference because
the generated path failed (too long?). I made a mistake reverting code and
I'll fix it

Le 7 oct. 2016 19:49, "Pierre Villard"  a écrit :

> *@pvillard31* requested changes on this pull request.
>
> Hi @mathiastiberghien , thanks for
> the additional work!
>
> Still some checkstyle issues:
>
> [WARNING] 
src/main/java/org/apache/nifi/controller/FlowController.java[1066:13] (blocks) 
LeftCurly: '{' should be on the previous line.
> [WARNING] 
src/test/java/org/apache/nifi/controller/TestFlowController.java[50] (imports) 
AvoidStarImport: Using the '.*' form of import should be avoided - 
org.apache.nifi.processor.*.
>
> Besides, could you remove the use of '_' in classes name and variables
> name. This is not very Java stylish :) (unless some specific cases such as
> final static variables). Also in your annotations, could you avoid 
starting
> methods name with a capital letter?
>
> I agree all of this could be checked through our checkstyle configuration
> (and I'll have a look at this) but I make those remarks to ensure some
> consistency in the code.
>
> Otherwise thanks for the unit tests!
> --
>
> In nifi-nar-bundles/nifi-framework-bundle/nifi-
> framework/nifi-framework-core/src/test/java/org/apache/nifi/
> controller/TestFlowController.java
> :
>
> > @@ -71,7 +74,7 @@
>
>  @Before
>  public void setup() {
> -System.setProperty(NiFiProperties.PROPERTIES_FILE_PATH, 
TestFlowController.class.getResource("/nifi.properties").getFile());
> +System.setProperty(NiFiProperties.PROPERTIES_FILE_PATH, 
FlowController.class.getResource("/nifi.properties").getFile());
>
> Is it really needed?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> , or 
mute
> the thread
> 

> .
>



> Allow components to provide default values for Yield Duration and Run Schedule
> --
>
> Key: NIFI-1526
> URL: https://issues.apache.org/jira/browse/NIFI-1526
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Bryan Bende
>Priority: Minor
>
> It would be nice for developers of processors (and maybe reporting tasks and 
> controller services) to be able to specify a default value for Yield duration 
> and Run Schedule.
> Currently Yield defaults to 1 second and Run Schedule defaults to 0 seconds. 
> There may be cases where these are not the best default values and the 
> developer wants to start off with better defaults, still allowing the user to 
> tune as needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1107: origin/NIFI-1526

[mathiastiberghien]

Github user mathiastiberghien commented on the issue:

https://github.com/apache/nifi/pull/1107
  
I'll correct styles (I'm really newbie to java and open source).
I add issues to launch the styles tests so  I hope I won't miss anything.
Sorry for the undesired change. I had to put a hard code reference because
the generated path failed (too long?). I made a mistake reverting code and
I'll fix it

Le 7 oct. 2016 19:49, "Pierre Villard"  a écrit :

> *@pvillard31* requested changes on this pull request.
>
> Hi @mathiastiberghien , thanks for
> the additional work!
>
> Still some checkstyle issues:
>
> [WARNING] 
src/main/java/org/apache/nifi/controller/FlowController.java[1066:13] (blocks) 
LeftCurly: '{' should be on the previous line.
> [WARNING] 
src/test/java/org/apache/nifi/controller/TestFlowController.java[50] (imports) 
AvoidStarImport: Using the '.*' form of import should be avoided - 
org.apache.nifi.processor.*.
>
> Besides, could you remove the use of '_' in classes name and variables
> name. This is not very Java stylish :) (unless some specific cases such as
> final static variables). Also in your annotations, could you avoid 
starting
> methods name with a capital letter?
>
> I agree all of this could be checked through our checkstyle configuration
> (and I'll have a look at this) but I make those remarks to ensure some
> consistency in the code.
>
> Otherwise thanks for the unit tests!
> --
>
> In nifi-nar-bundles/nifi-framework-bundle/nifi-
> framework/nifi-framework-core/src/test/java/org/apache/nifi/
> controller/TestFlowController.java
> :
>
> > @@ -71,7 +74,7 @@
>
>  @Before
>  public void setup() {
> -System.setProperty(NiFiProperties.PROPERTIES_FILE_PATH, 
TestFlowController.class.getResource("/nifi.properties").getFile());
> +System.setProperty(NiFiProperties.PROPERTIES_FILE_PATH, 
FlowController.class.getResource("/nifi.properties").getFile());
>
> Is it really needed?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> , or 
mute
> the thread
> 

> .
>



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-1526) Allow components to provide default values for Yield Duration and Run Schedule

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1752#comment-1752
 ] 

ASF GitHub Bot commented on NIFI-1526:
--

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1107#discussion_r82437230
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/test/java/org/apache/nifi/controller/TestFlowController.java
 ---
@@ -71,7 +74,7 @@
 
 @Before
 public void setup() {
-System.setProperty(NiFiProperties.PROPERTIES_FILE_PATH, 
TestFlowController.class.getResource("/nifi.properties").getFile());
+System.setProperty(NiFiProperties.PROPERTIES_FILE_PATH, 
FlowController.class.getResource("/nifi.properties").getFile());
--- End diff --

Is it really needed?


> Allow components to provide default values for Yield Duration and Run Schedule
> --
>
> Key: NIFI-1526
> URL: https://issues.apache.org/jira/browse/NIFI-1526
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Bryan Bende
>Priority: Minor
>
> It would be nice for developers of processors (and maybe reporting tasks and 
> controller services) to be able to specify a default value for Yield duration 
> and Run Schedule.
> Currently Yield defaults to 1 second and Run Schedule defaults to 0 seconds. 
> There may be cases where these are not the best default values and the 
> developer wants to start off with better defaults, still allowing the user to 
> tune as needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1107: origin/NIFI-1526

[pvillard31]

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1107#discussion_r82437230
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/test/java/org/apache/nifi/controller/TestFlowController.java
 ---
@@ -71,7 +74,7 @@
 
 @Before
 public void setup() {
-System.setProperty(NiFiProperties.PROPERTIES_FILE_PATH, 
TestFlowController.class.getResource("/nifi.properties").getFile());
+System.setProperty(NiFiProperties.PROPERTIES_FILE_PATH, 
FlowController.class.getResource("/nifi.properties").getFile());
--- End diff --

Is it really needed?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2874) StreamDemarcator can return wrong data for token

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1751#comment-1751
 ] 

ASF GitHub Bot commented on NIFI-2874:
--

Github user mosermw commented on the issue:

https://github.com/apache/nifi/pull/1114
  
reviewing


> StreamDemarcator can return wrong data for token
> 
>
> Key: NIFI-2874
> URL: https://issues.apache.org/jira/browse/NIFI-2874
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Critical
> Fix For: 1.1.0, 0.7.1
>
>
> There is a case where StreamDemarcator can return the wrong data for a token. 
> If a token ends at the end of the buffer, and the next token is smaller than 
> the previous, it can result in the next token keeping part of the buffer's 
> content. The code below is a unit test that exposes this:
> {code}
> @Test
> public void testOnBufferSplitNoTrailingDelimiter() throws IOException {
> final byte[] inputData = "Yes\nNo".getBytes(StandardCharsets.UTF_8);
> ByteArrayInputStream is = new ByteArrayInputStream(inputData);
> StreamDemarcator scanner = new StreamDemarcator(is, "\n".getBytes(), 
> 1000, 3);
> final byte[] first = scanner.nextToken();
> final byte[] second = scanner.nextToken();
> assertNotNull(first);
> assertNotNull(second);
> assertArrayEquals(first, new byte[] {'Y', 'e', 's'});
> assertArrayEquals(second, new byte[] {'N', 'o'});
> }
> {code}
> In this case, the second token, which should be 'No' comes back as 'Nos' 
> because it contains the 's' from the previous token.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1116: NIFI-2851 initial comit of perf improvements on Spl...

[olegz]

GitHub user olegz opened a pull request:

https://github.com/apache/nifi/pull/1116

NIFI-2851 initial comit of perf improvements on SplitText

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/olegz/nifi NIFI-2851

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1116.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1116


commit 0818d8689cb474bb1ad81dcd672cb0e31078825e
Author: Oleg Zhurakousky 
Date:   2016-10-07T15:37:32Z

NIFI-2851 initial comit of perf improvements on SplitText




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2851) Improve performance of SplitText

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1414#comment-1414
 ] 

ASF GitHub Bot commented on NIFI-2851:
--

GitHub user olegz opened a pull request:

https://github.com/apache/nifi/pull/1116

NIFI-2851 initial comit of perf improvements on SplitText

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/olegz/nifi NIFI-2851

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1116.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1116


commit 0818d8689cb474bb1ad81dcd672cb0e31078825e
Author: Oleg Zhurakousky 
Date:   2016-10-07T15:37:32Z

NIFI-2851 initial comit of perf improvements on SplitText




> Improve performance of SplitText
> 
>
> Key: NIFI-2851
> URL: https://issues.apache.org/jira/browse/NIFI-2851
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Oleg Zhurakousky
> Fix For: 1.1.0
>
>
> SplitText is fairly CPU-intensive and quite slow. A simple flow that splits a 
> 1.4 million line text file into 5k line chunks and then splits those 5k line 
> chunks into 1 line chunks is only capable of pushing through about 10k lines 
> per second. This equates to about 10 MB/sec. JVisualVM shows that the 
> majority of the time is spent in the locateSplitPoint() method. Isolating 
> this code and inspecting how it works, and using some micro-benchmarking, it 
> appears that if we refactor the calls to InputStream.read() to instead read 
> into a byte array, we can improve performance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2851) Improve performance of SplitText

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1425#comment-1425
 ] 

ASF GitHub Bot commented on NIFI-2851:
--

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1116#discussion_r82414967
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestSplitText.java
 ---
@@ -798,6 +798,7 @@ public void testWithSplitThatStartsWithNewLine() {
 }
 
 @Test
+@Ignore // temporary, fixing it
--- End diff --

@olegz did you overlook this?


> Improve performance of SplitText
> 
>
> Key: NIFI-2851
> URL: https://issues.apache.org/jira/browse/NIFI-2851
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Oleg Zhurakousky
> Fix For: 1.1.0
>
>
> SplitText is fairly CPU-intensive and quite slow. A simple flow that splits a 
> 1.4 million line text file into 5k line chunks and then splits those 5k line 
> chunks into 1 line chunks is only capable of pushing through about 10k lines 
> per second. This equates to about 10 MB/sec. JVisualVM shows that the 
> majority of the time is spent in the locateSplitPoint() method. Isolating 
> this code and inspecting how it works, and using some micro-benchmarking, it 
> appears that if we refactor the calls to InputStream.read() to instead read 
> into a byte array, we can improve performance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1116: NIFI-2851 initial comit of perf improvements on Spl...

[markap14]

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1116#discussion_r82414967
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestSplitText.java
 ---
@@ -798,6 +798,7 @@ public void testWithSplitThatStartsWithNewLine() {
 }
 
 @Test
+@Ignore // temporary, fixing it
--- End diff --

@olegz did you overlook this?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-2851) Improve performance of SplitText

[Oleg Zhurakousky (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Zhurakousky updated NIFI-2851:
---
Status: Patch Available  (was: Open)

> Improve performance of SplitText
> 
>
> Key: NIFI-2851
> URL: https://issues.apache.org/jira/browse/NIFI-2851
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Oleg Zhurakousky
> Fix For: 1.1.0
>
>
> SplitText is fairly CPU-intensive and quite slow. A simple flow that splits a 
> 1.4 million line text file into 5k line chunks and then splits those 5k line 
> chunks into 1 line chunks is only capable of pushing through about 10k lines 
> per second. This equates to about 10 MB/sec. JVisualVM shows that the 
> majority of the time is spent in the locateSplitPoint() method. Isolating 
> this code and inspecting how it works, and using some micro-benchmarking, it 
> appears that if we refactor the calls to InputStream.read() to instead read 
> into a byte array, we can improve performance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-2875) Provide a Caching FlowFileQueue that will cache a small number of FlowFiles to improve performance

[Mark Payne (JIRA)]

Mark Payne created NIFI-2875:


 Summary: Provide a Caching FlowFileQueue that will cache a small 
number of FlowFiles to improve performance
 Key: NIFI-2875
 URL: https://issues.apache.org/jira/browse/NIFI-2875
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework
Reporter: Mark Payne


Some Processors expect to pull a very large number of FlowFiles at a very fast 
rate. When this happens, there can be a great deal of contention of the 
StandardFlowFileQueue. We should create a CachingFlowFileQueue that will wrap 
another FlowFileQueue. When poll() is called, it will call poll(int) on the 
underlying FlowFileQueue to poll some number of FlowFiles (10 or 100, perhaps). 
It can then cache these FlowFiles so that subsequent calls to poll() returns 
the cached FlowFiles. This avoid a great deal of lock contention on 
StandardFlowFileQueue because that queue must perform a significant amount of 
work within its lock.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2850) Provide ability for a FlowFile to be migrated from one Process Session to another

[Mark Payne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Payne updated NIFI-2850:
-
Status: Patch Available  (was: Open)

> Provide ability for a FlowFile to be migrated from one Process Session to 
> another
> -
>
> Key: NIFI-2850
> URL: https://issues.apache.org/jira/browse/NIFI-2850
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.1.0
>
>
> Currently, the MergeContent processor creates a separate ProcessSession for 
> each FlowFile that it pulls. This is done so that we can ensure that we can 
> commit all Process Sessions when a bin is full. Unfortunately, this means 
> that MergeContent is required to call ProcessSession.get() many times, which 
> adds a lot of contention on the FlowFile Queue. If we allow FlowFiles to be 
> migrated from 1 session to another, we can have a session per bin, and then 
> use ProcessSession.get(100) to greatly reduce lock contention. This will 
> likely have benefits in other processors as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2850) Provide ability for a FlowFile to be migrated from one Process Session to another

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1396#comment-1396
 ] 

ASF GitHub Bot commented on NIFI-2850:
--

GitHub user markap14 opened a pull request:

https://github.com/apache/nifi/pull/1115

NIFI-2850: Added a migrate() method to ProcessSession and refactored …

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.

…BinFiles and MergeContent to use it

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/markap14/nifi NIFI-2850

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1115.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1115


commit d939a3d64fcc97a80322f292d13e3d61485a1b3e
Author: Mark Payne 
Date:   2016-09-29T18:41:35Z

NIFI-2850: Added a migrate() method to ProcessSession and refactored 
BinFiles and MergeContent to use it




> Provide ability for a FlowFile to be migrated from one Process Session to 
> another
> -
>
> Key: NIFI-2850
> URL: https://issues.apache.org/jira/browse/NIFI-2850
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.1.0
>
>
> Currently, the MergeContent processor creates a separate ProcessSession for 
> each FlowFile that it pulls. This is done so that we can ensure that we can 
> commit all Process Sessions when a bin is full. Unfortunately, this means 
> that MergeContent is required to call ProcessSession.get() many times, which 
> adds a lot of contention on the FlowFile Queue. If we allow FlowFiles to be 
> migrated from 1 session to another, we can have a session per bin, and then 
> use ProcessSession.get(100) to greatly reduce lock contention. This will 
> likely have benefits in other processors as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1115: NIFI-2850: Added a migrate() method to ProcessSessi...

[markap14]

GitHub user markap14 opened a pull request:

https://github.com/apache/nifi/pull/1115

NIFI-2850: Added a migrate() method to ProcessSession and refactored …

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.

…BinFiles and MergeContent to use it

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/markap14/nifi NIFI-2850

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1115.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1115


commit d939a3d64fcc97a80322f292d13e3d61485a1b3e
Author: Mark Payne 
Date:   2016-09-29T18:41:35Z

NIFI-2850: Added a migrate() method to ProcessSession and refactored 
BinFiles and MergeContent to use it




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (NIFI-2691) nifi.properties and admin guide refer to kerberos "principle" instead of "principal"

[Andrew Lim (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Lim resolved NIFI-2691.
--
Resolution: Fixed

> nifi.properties and admin guide refer to kerberos "principle" instead of 
> "principal"
> 
>
> Key: NIFI-2691
> URL: https://issues.apache.org/jira/browse/NIFI-2691
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework, Documentation & Website
>Affects Versions: 1.0.0
>Reporter: Andrew Lim
>Assignee: Andrew Lim
>Priority: Minor
>  Labels: beginner, kerberos
> Fix For: 1.1.0
>
>
> Need to change "principle" to "principal" in the following places:
> In nifi.properties:
> kerberos service *principle*
> kerberos spnego *principle*
> In Admin Guide for the description of the nifi.kerberos.krb5.file property:
> At this time, only a single krb5 file is allowed to be specified per NiFi 
> instance, so this property is configured here to support SPNEGO and service 
> *principles* rather than in individual Processors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (NIFI-2851) Improve performance of SplitText

[Oleg Zhurakousky (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Zhurakousky reassigned NIFI-2851:
--

Assignee: Oleg Zhurakousky  (was: Mark Payne)

> Improve performance of SplitText
> 
>
> Key: NIFI-2851
> URL: https://issues.apache.org/jira/browse/NIFI-2851
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Oleg Zhurakousky
> Fix For: 1.1.0
>
>
> SplitText is fairly CPU-intensive and quite slow. A simple flow that splits a 
> 1.4 million line text file into 5k line chunks and then splits those 5k line 
> chunks into 1 line chunks is only capable of pushing through about 10k lines 
> per second. This equates to about 10 MB/sec. JVisualVM shows that the 
> majority of the time is spent in the locateSplitPoint() method. Isolating 
> this code and inspecting how it works, and using some micro-benchmarking, it 
> appears that if we refactor the calls to InputStream.read() to instead read 
> into a byte array, we can improve performance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2874) StreamDemarcator can return wrong data for token

[Mark Payne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Payne updated NIFI-2874:
-
Status: Patch Available  (was: Open)

> StreamDemarcator can return wrong data for token
> 
>
> Key: NIFI-2874
> URL: https://issues.apache.org/jira/browse/NIFI-2874
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Critical
> Fix For: 1.1.0, 0.7.1
>
>
> There is a case where StreamDemarcator can return the wrong data for a token. 
> If a token ends at the end of the buffer, and the next token is smaller than 
> the previous, it can result in the next token keeping part of the buffer's 
> content. The code below is a unit test that exposes this:
> {code}
> @Test
> public void testOnBufferSplitNoTrailingDelimiter() throws IOException {
> final byte[] inputData = "Yes\nNo".getBytes(StandardCharsets.UTF_8);
> ByteArrayInputStream is = new ByteArrayInputStream(inputData);
> StreamDemarcator scanner = new StreamDemarcator(is, "\n".getBytes(), 
> 1000, 3);
> final byte[] first = scanner.nextToken();
> final byte[] second = scanner.nextToken();
> assertNotNull(first);
> assertNotNull(second);
> assertArrayEquals(first, new byte[] {'Y', 'e', 's'});
> assertArrayEquals(second, new byte[] {'N', 'o'});
> }
> {code}
> In this case, the second token, which should be 'No' comes back as 'Nos' 
> because it contains the 's' from the previous token.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2874) StreamDemarcator can return wrong data for token

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1135#comment-1135
 ] 

ASF GitHub Bot commented on NIFI-2874:
--

GitHub user markap14 opened a pull request:

https://github.com/apache/nifi/pull/1114

NIFI-2874: Ensure that when reading more data from an InputStream the…

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.

… StreamDemarcator appropriately updates the max index that can be read 
from the buffer

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/markap14/nifi NIFI-2874

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1114.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1114


commit 0f1eb298cc42933711f60312b57a7d66004c7d13
Author: Mark Payne 
Date:   2016-10-07T13:52:03Z

NIFI-2874: Ensure that when reading more data from an InputStream the 
StreamDemarcator appropriately updates the max index that can be read from the 
buffer




> StreamDemarcator can return wrong data for token
> 
>
> Key: NIFI-2874
> URL: https://issues.apache.org/jira/browse/NIFI-2874
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Critical
> Fix For: 1.1.0, 0.7.1
>
>
> There is a case where StreamDemarcator can return the wrong data for a token. 
> If a token ends at the end of the buffer, and the next token is smaller than 
> the previous, it can result in the next token keeping part of the buffer's 
> content. The code below is a unit test that exposes this:
> {code}
> @Test
> public void testOnBufferSplitNoTrailingDelimiter() throws IOException {
> final byte[] inputData = "Yes\nNo".getBytes(StandardCharsets.UTF_8);
> ByteArrayInputStream is = new ByteArrayInputStream(inputData);
> StreamDemarcator scanner = new StreamDemarcator(is, "\n".getBytes(), 
> 1000, 3);
> final byte[] first = scanner.nextToken();
> final byte[] second = scanner.nextToken();
> assertNotNull(first);
> assertNotNull(second);
> assertArrayEquals(first, new byte[] {'Y', 'e', 's'});
> assertArrayEquals(second, new byte[] {'N', 'o'});
> }
> {code}
> In this case, the second token, which should be 'No' comes back as 'Nos' 
> because it contains the 's' from the previous token.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1114: NIFI-2874: Ensure that when reading more data from ...

[markap14]

GitHub user markap14 opened a pull request:

https://github.com/apache/nifi/pull/1114

NIFI-2874: Ensure that when reading more data from an InputStream the…

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.

… StreamDemarcator appropriately updates the max index that can be read 
from the buffer

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/markap14/nifi NIFI-2874

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1114.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1114


commit 0f1eb298cc42933711f60312b57a7d66004c7d13
Author: Mark Payne 
Date:   2016-10-07T13:52:03Z

NIFI-2874: Ensure that when reading more data from an InputStream the 
StreamDemarcator appropriately updates the max index that can be read from the 
buffer




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #1112: NIFI-2799: AWS Credentials for Assume Role Need Proxy

[ktseytlin]

Github user ktseytlin commented on the issue:

https://github.com/apache/nifi/pull/1112
  
This is obviously a pretty awful drawing I threw together... but it shows 
what AWS is like in an enterprise. For every single service I ever want to 
execute, I need to go through a proxy. Hence why this bug fix is needed. The 
only situation in which this wouldn't occur is if the enterprise specifically 
opened up the endpoints to each service, which would allow it to avoid going 
through the proxy. 


![image](https://cloud.githubusercontent.com/assets/10350664/19191905/4e30ecee-8c72-11e6-9cab-3f42caee5f4f.png)

@jvwing Hope this is helpful for regarding our previous conversation about 
proxy things :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2799) AWS Credentials for Assume Role Need Proxy

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1117#comment-1117
 ] 

ASF GitHub Bot commented on NIFI-2799:
--

Github user ktseytlin commented on the issue:

https://github.com/apache/nifi/pull/1112
  
This is obviously a pretty awful drawing I threw together... but it shows 
what AWS is like in an enterprise. For every single service I ever want to 
execute, I need to go through a proxy. Hence why this bug fix is needed. The 
only situation in which this wouldn't occur is if the enterprise specifically 
opened up the endpoints to each service, which would allow it to avoid going 
through the proxy. 


![image](https://cloud.githubusercontent.com/assets/10350664/19191905/4e30ecee-8c72-11e6-9cab-3f42caee5f4f.png)

@jvwing Hope this is helpful for regarding our previous conversation about 
proxy things :)


> AWS Credentials for Assume Role Need Proxy
> --
>
> Key: NIFI-2799
> URL: https://issues.apache.org/jira/browse/NIFI-2799
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Keren Tseytlin
>Assignee: James Wing
>Priority: Minor
> Fix For: 1.1.0
>
>
> As a user of Nifi, when I want to enable cross account fetching of S3 objects 
> I need the proxy variables to be set in order to generate temporary AWS 
> tokens for STS:AssumeRole.
> Within some enterprise environments, it is necessary to set the proxy 
> variables prior to running AssumeRole methods. Without this being set, the 
> machine in VPC A times out on generating temporary keys and is unable to 
> assume a role as a machine in VPC B. 
> This ticket arose from this conversation: 
> http://apache-nifi-developer-list.39713.n7.nabble.com/Nifi-Cross-Account-Download-With-A-Profile-Flag-td13232.html#a13252
> Goal: There are files stored in an S3 bucket in VPC B. My Nifi machines are 
> in VPC A. I want Nifi to be able to get those files from VPC B. VPC A and VPC 
> B need to be communicating in the FetchS3Object component.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi-minifi-cpp issue #18: MINIFI-34 - attempt to progress the CMake environ...

[apiri]

Github user apiri commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/18
  
Hmm, okay.  Good information and agree on supporting the listed 
environments.  I think I can work around the INTERFACE usage.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2799) AWS Credentials for Assume Role Need Proxy

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1095#comment-1095
 ] 

ASF GitHub Bot commented on NIFI-2799:
--

Github user ktseytlin commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1112#discussion_r82389180
  
--- Diff: 
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/credentials/provider/factory/CredentialPropertyDescriptors.java
 ---
@@ -44,7 +45,7 @@
 .allowableValues("true", "false")
 .defaultValue("false")
 .description("If true, uses the Default Credential chain, 
including EC2 instance profiles or roles, " +
-"environment variables, default user credentials, etc.")
+"environment variables, default user credentials, 
etc.")
--- End diff --

Yes, it's just a formatting change to make it clear that the break in the 
string means that it's still part of the description.


> AWS Credentials for Assume Role Need Proxy
> --
>
> Key: NIFI-2799
> URL: https://issues.apache.org/jira/browse/NIFI-2799
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Keren Tseytlin
>Assignee: James Wing
>Priority: Minor
> Fix For: 1.1.0
>
>
> As a user of Nifi, when I want to enable cross account fetching of S3 objects 
> I need the proxy variables to be set in order to generate temporary AWS 
> tokens for STS:AssumeRole.
> Within some enterprise environments, it is necessary to set the proxy 
> variables prior to running AssumeRole methods. Without this being set, the 
> machine in VPC A times out on generating temporary keys and is unable to 
> assume a role as a machine in VPC B. 
> This ticket arose from this conversation: 
> http://apache-nifi-developer-list.39713.n7.nabble.com/Nifi-Cross-Account-Download-With-A-Profile-Flag-td13232.html#a13252
> Goal: There are files stored in an S3 bucket in VPC B. My Nifi machines are 
> in VPC A. I want Nifi to be able to get those files from VPC B. VPC A and VPC 
> B need to be communicating in the FetchS3Object component.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2799) AWS Credentials for Assume Role Need Proxy

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1096#comment-1096
 ] 

ASF GitHub Bot commented on NIFI-2799:
--

Github user ktseytlin commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1112#discussion_r82389187
  
--- Diff: 
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/credentials/provider/factory/CredentialPropertyDescriptors.java
 ---
@@ -153,6 +154,29 @@
 .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
 .sensitive(false)
 .description("External ID for cross-account access. This is 
used in conjunction with role arn, " +
-"role name, and optional session time out")
+"role name, and optional session time out")
--- End diff --

Yes, it's just a formatting change to make it clear that the break in the 
string means that it's still part of the description.


> AWS Credentials for Assume Role Need Proxy
> --
>
> Key: NIFI-2799
> URL: https://issues.apache.org/jira/browse/NIFI-2799
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Keren Tseytlin
>Assignee: James Wing
>Priority: Minor
> Fix For: 1.1.0
>
>
> As a user of Nifi, when I want to enable cross account fetching of S3 objects 
> I need the proxy variables to be set in order to generate temporary AWS 
> tokens for STS:AssumeRole.
> Within some enterprise environments, it is necessary to set the proxy 
> variables prior to running AssumeRole methods. Without this being set, the 
> machine in VPC A times out on generating temporary keys and is unable to 
> assume a role as a machine in VPC B. 
> This ticket arose from this conversation: 
> http://apache-nifi-developer-list.39713.n7.nabble.com/Nifi-Cross-Account-Download-With-A-Profile-Flag-td13232.html#a13252
> Goal: There are files stored in an S3 bucket in VPC B. My Nifi machines are 
> in VPC A. I want Nifi to be able to get those files from VPC B. VPC A and VPC 
> B need to be communicating in the FetchS3Object component.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1112: NIFI-2799: AWS Credentials for Assume Role Need Pro...

[ktseytlin]

Github user ktseytlin commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1112#discussion_r82389180
  
--- Diff: 
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/credentials/provider/factory/CredentialPropertyDescriptors.java
 ---
@@ -44,7 +45,7 @@
 .allowableValues("true", "false")
 .defaultValue("false")
 .description("If true, uses the Default Credential chain, 
including EC2 instance profiles or roles, " +
-"environment variables, default user credentials, etc.")
+"environment variables, default user credentials, 
etc.")
--- End diff --

Yes, it's just a formatting change to make it clear that the break in the 
string means that it's still part of the description.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1112: NIFI-2799: AWS Credentials for Assume Role Need Pro...

[ktseytlin]

Github user ktseytlin commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1112#discussion_r82389187
  
--- Diff: 
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/credentials/provider/factory/CredentialPropertyDescriptors.java
 ---
@@ -153,6 +154,29 @@
 .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
 .sensitive(false)
 .description("External ID for cross-account access. This is 
used in conjunction with role arn, " +
-"role name, and optional session time out")
+"role name, and optional session time out")
--- End diff --

Yes, it's just a formatting change to make it clear that the break in the 
string means that it's still part of the description.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2799) AWS Credentials for Assume Role Need Proxy

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1088#comment-1088
 ] 

ASF GitHub Bot commented on NIFI-2799:
--

Github user ktseytlin commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1112#discussion_r82388965
  
--- Diff: 
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/credentials/provider/factory/CredentialPropertyDescriptors.java
 ---
@@ -18,6 +18,7 @@
 
 import org.apache.nifi.components.PropertyDescriptor;
 import org.apache.nifi.processor.util.StandardValidators;
+import org.joda.time.DateTime;
--- End diff --

Yep, will remove. I actually don't remember adding this, my bad.


> AWS Credentials for Assume Role Need Proxy
> --
>
> Key: NIFI-2799
> URL: https://issues.apache.org/jira/browse/NIFI-2799
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Keren Tseytlin
>Assignee: James Wing
>Priority: Minor
> Fix For: 1.1.0
>
>
> As a user of Nifi, when I want to enable cross account fetching of S3 objects 
> I need the proxy variables to be set in order to generate temporary AWS 
> tokens for STS:AssumeRole.
> Within some enterprise environments, it is necessary to set the proxy 
> variables prior to running AssumeRole methods. Without this being set, the 
> machine in VPC A times out on generating temporary keys and is unable to 
> assume a role as a machine in VPC B. 
> This ticket arose from this conversation: 
> http://apache-nifi-developer-list.39713.n7.nabble.com/Nifi-Cross-Account-Download-With-A-Profile-Flag-td13232.html#a13252
> Goal: There are files stored in an S3 bucket in VPC B. My Nifi machines are 
> in VPC A. I want Nifi to be able to get those files from VPC B. VPC A and VPC 
> B need to be communicating in the FetchS3Object component.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1112: NIFI-2799: AWS Credentials for Assume Role Need Pro...

[ktseytlin]

Github user ktseytlin commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1112#discussion_r82388965
  
--- Diff: 
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/credentials/provider/factory/CredentialPropertyDescriptors.java
 ---
@@ -18,6 +18,7 @@
 
 import org.apache.nifi.components.PropertyDescriptor;
 import org.apache.nifi.processor.util.StandardValidators;
+import org.joda.time.DateTime;
--- End diff --

Yep, will remove. I actually don't remember adding this, my bad.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2874) StreamDemarcator can return wrong data for token

[Mark Payne (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1085#comment-1085
 ] 

Mark Payne commented on NIFI-2874:
--

This was addressed in the 1.1.0 baseline already via NIFI-2865, as it was found 
when testing the implementation of that ticket. So no PR is needed for the 1.x 
baseline. Will create a PR to address in the 0.x baseline.

> StreamDemarcator can return wrong data for token
> 
>
> Key: NIFI-2874
> URL: https://issues.apache.org/jira/browse/NIFI-2874
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Critical
> Fix For: 1.1.0, 0.7.1
>
>
> There is a case where StreamDemarcator can return the wrong data for a token. 
> If a token ends at the end of the buffer, and the next token is smaller than 
> the previous, it can result in the next token keeping part of the buffer's 
> content. The code below is a unit test that exposes this:
> {code}
> @Test
> public void testOnBufferSplitNoTrailingDelimiter() throws IOException {
> final byte[] inputData = "Yes\nNo".getBytes(StandardCharsets.UTF_8);
> ByteArrayInputStream is = new ByteArrayInputStream(inputData);
> StreamDemarcator scanner = new StreamDemarcator(is, "\n".getBytes(), 
> 1000, 3);
> final byte[] first = scanner.nextToken();
> final byte[] second = scanner.nextToken();
> assertNotNull(first);
> assertNotNull(second);
> assertArrayEquals(first, new byte[] {'Y', 'e', 's'});
> assertArrayEquals(second, new byte[] {'N', 'o'});
> }
> {code}
> In this case, the second token, which should be 'No' comes back as 'Nos' 
> because it contains the 's' from the previous token.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-2874) StreamDemarcator can return wrong data for token

[Mark Payne (JIRA)]

Mark Payne created NIFI-2874:


 Summary: StreamDemarcator can return wrong data for token
 Key: NIFI-2874
 URL: https://issues.apache.org/jira/browse/NIFI-2874
 Project: Apache NiFi
  Issue Type: Bug
  Components: Extensions
Reporter: Mark Payne
Assignee: Mark Payne
Priority: Critical
 Fix For: 1.1.0, 0.7.1


There is a case where StreamDemarcator can return the wrong data for a token. 
If a token ends at the end of the buffer, and the next token is smaller than 
the previous, it can result in the next token keeping part of the buffer's 
content. The code below is a unit test that exposes this:

{code}
@Test
public void testOnBufferSplitNoTrailingDelimiter() throws IOException {
final byte[] inputData = "Yes\nNo".getBytes(StandardCharsets.UTF_8);
ByteArrayInputStream is = new ByteArrayInputStream(inputData);
StreamDemarcator scanner = new StreamDemarcator(is, "\n".getBytes(), 
1000, 3);

final byte[] first = scanner.nextToken();
final byte[] second = scanner.nextToken();
assertNotNull(first);
assertNotNull(second);

assertArrayEquals(first, new byte[] {'Y', 'e', 's'});
assertArrayEquals(second, new byte[] {'N', 'o'});
}
{code}

In this case, the second token, which should be 'No' comes back as 'Nos' 
because it contains the 's' from the previous token.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1526) Allow components to provide default values for Yield Duration and Run Schedule

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1049#comment-1049
 ] 

ASF GitHub Bot commented on NIFI-1526:
--

Github user mathiastiberghien commented on the issue:

https://github.com/apache/nifi/pull/1107
  
I've been able to find the problem with Dummy processors (the class were 
declared in the test file which created an InstanciationException).
So I commited the code with style corrections, exceptions added to Log 
methods to have the stack trace, and 2 tests added to the TestFlowController 
class testing each annotation


> Allow components to provide default values for Yield Duration and Run Schedule
> --
>
> Key: NIFI-1526
> URL: https://issues.apache.org/jira/browse/NIFI-1526
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Bryan Bende
>Priority: Minor
>
> It would be nice for developers of processors (and maybe reporting tasks and 
> controller services) to be able to specify a default value for Yield duration 
> and Run Schedule.
> Currently Yield defaults to 1 second and Run Schedule defaults to 0 seconds. 
> There may be cases where these are not the best default values and the 
> developer wants to start off with better defaults, still allowing the user to 
> tune as needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1107: origin/NIFI-1526

[mathiastiberghien]

Github user mathiastiberghien commented on the issue:

https://github.com/apache/nifi/pull/1107
  
I've been able to find the problem with Dummy processors (the class were 
declared in the test file which created an InstanciationException).
So I commited the code with style corrections, exceptions added to Log 
methods to have the stack trace, and 2 tests added to the TestFlowController 
class testing each annotation


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-1526) Allow components to provide default values for Yield Duration and Run Schedule

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554829#comment-15554829
 ] 

ASF GitHub Bot commented on NIFI-1526:
--

Github user mathiastiberghien commented on the issue:

https://github.com/apache/nifi/pull/1107
  
Hello,

Thanks for the comments.

I have a question about unit test: I need to test the method 
createProcessor of the FlowController so I guess I need to update the 
TestFlowController file adding a method that add some dummy processors and 
check the behavior.

 

How do I proceed to add my dummy processors class to the system, because 
createProcessor method requires the type and the identifier (which is?)

 

I’ve trie to create a dummy class processor and use class.forname method 
but it didn’t work

 

 

Mathias TIBERGHIEN

CTO

Cell. : +33(0)6 27 58 13 68

  

87 Bd Chanzy

93100 Montreuil

France

 

Tel : +33(0)1 42 87 16 57

www.code192.com  

 

From: Pierre Villard [mailto:notificati...@github.com] 
Sent: mercredi 5 octobre 2016 23:11
To: apache/nifi 
Cc: Mathias Tiberghien ; Author 

Subject: Re: [apache/nifi] origin/NIFI-1526 (#1107)

 

@pvillard31 requested changes on this pull request.

Few preliminary stylish remarks.
I would also love to see some unit tests in nifi-framework-core with dummy 
processors to confirm the good behavior of the annotations.
In any case, this will be a useful improvement, thanks for contributing!

  _  

In 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/FlowController.java
  :

> @@ -1059,6 +1060,42 @@ public ProcessorNode createProcessor(final String 
type, String id, final boolean
 final LogRepository logRepository = 
LogRepositoryFactory.getRepository(id);
 
logRepository.addObserver(StandardProcessorNode.BULLETIN_OBSERVER_ID, 
LogLevel.WARN, new ProcessorLogObserver(getBulletinRepository(), procNode));
 
+try {
+
+final Class procClass = processor.getClass();
+if(procClass.isAnnotationPresent(DefaultSettings.class))
+{

There are checkstyle violations to fix:

[WARNING] 
src/main/java/org/apache/nifi/annotation/configuration/DefaultSchedule.java[5] 
(imports) AvoidStarImport: Using the '.*' form of import should be avoided - 
java.lang.annotation.*.
[WARNING] 
src/main/java/org/apache/nifi/annotation/configuration/DefaultSettings.java[5] 
(imports) AvoidStarImport: Using the '.*' form of import should be avoided - 
java.lang.annotation.*.
[WARNING] 
src/main/java/org/apache/nifi/controller/FlowController.java[1067:13] (blocks) 
LeftCurly: '{' should be on the previous line.
[WARNING] 
src/main/java/org/apache/nifi/controller/FlowController.java[1071:17] (blocks) 
RightCurly: '}' should be on the same line.
[WARNING] 
src/main/java/org/apache/nifi/controller/FlowController.java[1073:17] (blocks) 
LeftCurly: '{' should be on the previous line.
[WARNING] 
src/main/java/org/apache/nifi/controller/FlowController.java[1079:17] (blocks) 
RightCurly: '}' should be on the same line.
[WARNING] 
src/main/java/org/apache/nifi/controller/FlowController.java[1081:17] (blocks) 
LeftCurly: '{' should be on the previous line.
[WARNING] 
src/main/java/org/apache/nifi/controller/FlowController.java[1086:17] (blocks) 
RightCurly: '}' should be on the same line.
[WARNING] 
src/main/java/org/apache/nifi/controller/FlowController.java[1088:17] (blocks) 
LeftCurly: '{' should be on the previous line.
[WARNING] 
src/main/java/org/apache/nifi/controller/FlowController.java[1093:9] (blocks) 
RightCurly: '}' should be on the same line.
[WARNING] 
src/main/java/org/apache/nifi/controller/FlowController.java[1095:9] (blocks) 
LeftCurly: '{' should be on the previous line.
[WARNING] 
src/main/java/org/apache/nifi/controller/StandardProcessorNode.java[191:9] 
(blocks) LeftCurly: '{' should be on the previous line.
[WARNING] 
src/main/java/org/apache/nifi/controller/StandardProcessorNode.java[194:13] 
(blocks) LeftCurly: '{' should be on the previous line.
[WARNING] 
src/main/java/org/apache/nifi/controller/StandardProcessorNode.java[197:17] 
(blocks) LeftCurly: '{' should be on the previous line.
[WARNING] 
src/main/java/org/apache/nifi/controller/StandardProcessorNode.java[199:17] 
(blocks) RightCurly: '}' should be on the same line.
[WARNING] 

[GitHub] nifi issue #1107: origin/NIFI-1526

[mathiastiberghien]

Github user mathiastiberghien commented on the issue:

https://github.com/apache/nifi/pull/1107
  
Hello,

Thanks for the comments.

I have a question about unit test: I need to test the method 
createProcessor of the FlowController so I guess I need to update the 
TestFlowController file adding a method that add some dummy processors and 
check the behavior.

 

How do I proceed to add my dummy processors class to the system, because 
createProcessor method requires the type and the identifier (which is?)

 

I’ve trie to create a dummy class processor and use class.forname method 
but it didn’t work

 

 

Mathias TIBERGHIEN

CTO

Cell. : +33(0)6 27 58 13 68

  

87 Bd Chanzy

93100 Montreuil

France

 

Tel : +33(0)1 42 87 16 57

www.code192.com  

 

From: Pierre Villard [mailto:notificati...@github.com] 
Sent: mercredi 5 octobre 2016 23:11
To: apache/nifi 
Cc: Mathias Tiberghien ; Author 

Subject: Re: [apache/nifi] origin/NIFI-1526 (#1107)

 

@pvillard31 requested changes on this pull request.

Few preliminary stylish remarks.
I would also love to see some unit tests in nifi-framework-core with dummy 
processors to confirm the good behavior of the annotations.
In any case, this will be a useful improvement, thanks for contributing!

  _  

In 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/FlowController.java
  :

> @@ -1059,6 +1060,42 @@ public ProcessorNode createProcessor(final String 
type, String id, final boolean
 final LogRepository logRepository = 
LogRepositoryFactory.getRepository(id);
 
logRepository.addObserver(StandardProcessorNode.BULLETIN_OBSERVER_ID, 
LogLevel.WARN, new ProcessorLogObserver(getBulletinRepository(), procNode));
 
+try {
+
+final Class procClass = processor.getClass();
+if(procClass.isAnnotationPresent(DefaultSettings.class))
+{

There are checkstyle violations to fix:

[WARNING] 
src/main/java/org/apache/nifi/annotation/configuration/DefaultSchedule.java[5] 
(imports) AvoidStarImport: Using the '.*' form of import should be avoided - 
java.lang.annotation.*.
[WARNING] 
src/main/java/org/apache/nifi/annotation/configuration/DefaultSettings.java[5] 
(imports) AvoidStarImport: Using the '.*' form of import should be avoided - 
java.lang.annotation.*.
[WARNING] 
src/main/java/org/apache/nifi/controller/FlowController.java[1067:13] (blocks) 
LeftCurly: '{' should be on the previous line.
[WARNING] 
src/main/java/org/apache/nifi/controller/FlowController.java[1071:17] (blocks) 
RightCurly: '}' should be on the same line.
[WARNING] 
src/main/java/org/apache/nifi/controller/FlowController.java[1073:17] (blocks) 
LeftCurly: '{' should be on the previous line.
[WARNING] 
src/main/java/org/apache/nifi/controller/FlowController.java[1079:17] (blocks) 
RightCurly: '}' should be on the same line.
[WARNING] 
src/main/java/org/apache/nifi/controller/FlowController.java[1081:17] (blocks) 
LeftCurly: '{' should be on the previous line.
[WARNING] 
src/main/java/org/apache/nifi/controller/FlowController.java[1086:17] (blocks) 
RightCurly: '}' should be on the same line.
[WARNING] 
src/main/java/org/apache/nifi/controller/FlowController.java[1088:17] (blocks) 
LeftCurly: '{' should be on the previous line.
[WARNING] 
src/main/java/org/apache/nifi/controller/FlowController.java[1093:9] (blocks) 
RightCurly: '}' should be on the same line.
[WARNING] 
src/main/java/org/apache/nifi/controller/FlowController.java[1095:9] (blocks) 
LeftCurly: '{' should be on the previous line.
[WARNING] 
src/main/java/org/apache/nifi/controller/StandardProcessorNode.java[191:9] 
(blocks) LeftCurly: '{' should be on the previous line.
[WARNING] 
src/main/java/org/apache/nifi/controller/StandardProcessorNode.java[194:13] 
(blocks) LeftCurly: '{' should be on the previous line.
[WARNING] 
src/main/java/org/apache/nifi/controller/StandardProcessorNode.java[197:17] 
(blocks) LeftCurly: '{' should be on the previous line.
[WARNING] 
src/main/java/org/apache/nifi/controller/StandardProcessorNode.java[199:17] 
(blocks) RightCurly: '}' should be on the same line.
[WARNING] 
src/main/java/org/apache/nifi/controller/StandardProcessorNode.java[201:17] 
(blocks) LeftCurly: '{' should be on the previous line.
[WARNING] 
src/main/java/org/apache/nifi/controller/StandardProcessorNode.java[205:17] 
(blocks) LeftCurly: '{' should be on the previous line.
[WARNING] 

[jira] [Commented] (NIFI-2855) NiFi Site-To-Site with port forwarding

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554239#comment-15554239
 ] 

ASF GitHub Bot commented on NIFI-2855:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1100


> NiFi Site-To-Site with port forwarding
> --
>
> Key: NIFI-2855
> URL: https://issues.apache.org/jira/browse/NIFI-2855
> Project: Apache NiFi
>  Issue Type: Improvement
>Affects Versions: 1.0.0
>Reporter: Bryan Rosander
>Assignee: Koji Kawamura
>
> It would be useful to be able to use port forwarding with NiFi Site-To-Site.  
> This would allow NiFi to appear externally to be listening on a privileged 
> port without having been granted elevated permissions.
> For example, an administrator could configure iptables to forward traffic 
> from port 443 to port 9443.  Then users could use NiFi at port 443.  This 
> provides more flexibility as far as firewall configuration is concerned.
> The above scenario causes problems with Site-To-Site though because in a 
> clustered scenario, the nodes will still advertise themselves with port 9443. 
>  This would prevent a Site-To-Site client from being able to talk to them 
> from outside the firewall.
> We need a way (probably a nifi property) to tell NiFi to listen on one port 
> (9443) and advertise another (443) for Site-To-Site purposes to enable this 
> usecase.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2855) NiFi Site-To-Site with port forwarding

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554237#comment-15554237
 ] 

ASF subversion and git services commented on NIFI-2855:
---

Commit 540ef63efa9d6b2af84f57fe4eae2f08e6dd1693 in nifi's branch 
refs/heads/master from [~ijokarumawak]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=540ef63 ]

NIFI-2855: Site-to-Site with port forwarding.

- Added following properties:
  - nifi.web.http.port.forwarding
  - nifi.web.https.port.forwarding

This closes #1100.

Signed-off-by: Koji Kawamura 


> NiFi Site-To-Site with port forwarding
> --
>
> Key: NIFI-2855
> URL: https://issues.apache.org/jira/browse/NIFI-2855
> Project: Apache NiFi
>  Issue Type: Improvement
>Affects Versions: 1.0.0
>Reporter: Bryan Rosander
>Assignee: Koji Kawamura
>
> It would be useful to be able to use port forwarding with NiFi Site-To-Site.  
> This would allow NiFi to appear externally to be listening on a privileged 
> port without having been granted elevated permissions.
> For example, an administrator could configure iptables to forward traffic 
> from port 443 to port 9443.  Then users could use NiFi at port 443.  This 
> provides more flexibility as far as firewall configuration is concerned.
> The above scenario causes problems with Site-To-Site though because in a 
> clustered scenario, the nodes will still advertise themselves with port 9443. 
>  This would prevent a Site-To-Site client from being able to talk to them 
> from outside the firewall.
> We need a way (probably a nifi property) to tell NiFi to listen on one port 
> (9443) and advertise another (443) for Site-To-Site purposes to enable this 
> usecase.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2855) NiFi Site-To-Site with port forwarding

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554236#comment-15554236
 ] 

ASF GitHub Bot commented on NIFI-2855:
--

Github user ijokarumawak commented on the issue:

https://github.com/apache/nifi/pull/1100
  
@brosander Thank you for reviewing and testing this. I will merge this to 
master following your +1.


> NiFi Site-To-Site with port forwarding
> --
>
> Key: NIFI-2855
> URL: https://issues.apache.org/jira/browse/NIFI-2855
> Project: Apache NiFi
>  Issue Type: Improvement
>Affects Versions: 1.0.0
>Reporter: Bryan Rosander
>Assignee: Koji Kawamura
>
> It would be useful to be able to use port forwarding with NiFi Site-To-Site.  
> This would allow NiFi to appear externally to be listening on a privileged 
> port without having been granted elevated permissions.
> For example, an administrator could configure iptables to forward traffic 
> from port 443 to port 9443.  Then users could use NiFi at port 443.  This 
> provides more flexibility as far as firewall configuration is concerned.
> The above scenario causes problems with Site-To-Site though because in a 
> clustered scenario, the nodes will still advertise themselves with port 9443. 
>  This would prevent a Site-To-Site client from being able to talk to them 
> from outside the firewall.
> We need a way (probably a nifi property) to tell NiFi to listen on one port 
> (9443) and advertise another (443) for Site-To-Site purposes to enable this 
> usecase.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1100: NIFI-2855: Site-to-Site with port forwarding.

[ijokarumawak]

Github user ijokarumawak commented on the issue:

https://github.com/apache/nifi/pull/1100
  
@brosander Thank you for reviewing and testing this. I will merge this to 
master following your +1.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---