[jira] [Created] (NIFI-1026) Add support for running as a service in Windows

[Brandon DeVries (JIRA)]

Brandon DeVries created NIFI-1026:
-

 Summary: Add support for running as a service in Windows
 Key: NIFI-1026
 URL: https://issues.apache.org/jira/browse/NIFI-1026
 Project: Apache NiFi
  Issue Type: Improvement
Reporter: Brandon DeVries


Currently, installing NiFi as a service is supported only for Linux and Mac OSX 
users.  Add support for Windows...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-810) Create Annotation that indicates that a Processor cannot be scheduled to run without an incoming connection

[Matt Gilman (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14948771#comment-14948771
 ] 

Matt Gilman commented on NIFI-810:
--

+1 - Relevant code changes look good. Behavior is as expected. Updated 
validation error is more concise.
0 - Extraneous formatting changes and sporadic ^M line breaks.

> Create Annotation that indicates that a Processor cannot be scheduled to run 
> without an incoming connection
> ---
>
> Key: NIFI-810
> URL: https://issues.apache.org/jira/browse/NIFI-810
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, if a Processor has no incoming connections but is started, it will 
> continually without ever accomplishing anything. We should have an 
> annotation, perhaps @RequiresInput, that indicates that the Processor should 
> not be scheduled to run unless it has an incoming connection.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-942) Create RouteText processor

[Joseph Percivall (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Percivall updated NIFI-942:
--
Attachment: (was: NIFI-942.patch)

> Create RouteText processor
> --
>
> Key: NIFI-942
> URL: https://issues.apache.org/jira/browse/NIFI-942
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Joseph Percivall
> Fix For: 0.4.0
>
> Attachments: NIFI-942.patch
>
>
> The idea is to route individual lines of a text file to different 
> relationships. This allows for splitting lines based on some criteria or 
> filtering out specific lines, and would be a much more convenient alternative 
> than RouteOnContent for textual data.
> A discussion for this took place on the users mailing list 
> (http://mail-archives.apache.org/mod_mbox/nifi-users/201509.mbox/%3CCAKpk5PxjszdX-NXMMf6Pcet4x7Y5GmrT7_jn9uyzS-h_a9TG3A%40mail.gmail.com%3E)
> The way that I could see this working is to have a few different properties:
> Routing Strategy:
> - Route each line to matching Property Name (default)
> - Route matching lines to 'matched' if all match
> - Route matching lines to 'matched' if any match
> - Route FlowFile to 'matched' if all lines match
> - Route FlowFile to 'matched' if any line matches
> A Match Strategy
> - Starts With
> - Ends With
> - Contains
> - Equals
> - Matches Regular Expression
> - Contains Regular Expression
> And then user-defined properties that indicate what to search each line of 
> text for.
> So to find lines that begin with the < character
> You would simply add a property named "Begins with Less Than" and set the 
> value to : <
> Then set the Match Strategy to Starts With
> And Routing Strategy to "Route each line to matching Property Name"
> Then, any line that begins with < will be routed to the Begins with Less Than 
> relationship.
> This would be a simple way to pull out any particular lines of interest in a 
> text file.
> I can see this being very useful for processing log files, CSV, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-942) Create RouteText processor

[Joseph Percivall (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Percivall updated NIFI-942:
--
Attachment: NIFI-942.patch

Forgot to pass in the charset to a getBytes call

> Create RouteText processor
> --
>
> Key: NIFI-942
> URL: https://issues.apache.org/jira/browse/NIFI-942
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Joseph Percivall
> Fix For: 0.4.0
>
> Attachments: NIFI-942.patch
>
>
> The idea is to route individual lines of a text file to different 
> relationships. This allows for splitting lines based on some criteria or 
> filtering out specific lines, and would be a much more convenient alternative 
> than RouteOnContent for textual data.
> A discussion for this took place on the users mailing list 
> (http://mail-archives.apache.org/mod_mbox/nifi-users/201509.mbox/%3CCAKpk5PxjszdX-NXMMf6Pcet4x7Y5GmrT7_jn9uyzS-h_a9TG3A%40mail.gmail.com%3E)
> The way that I could see this working is to have a few different properties:
> Routing Strategy:
> - Route each line to matching Property Name (default)
> - Route matching lines to 'matched' if all match
> - Route matching lines to 'matched' if any match
> - Route FlowFile to 'matched' if all lines match
> - Route FlowFile to 'matched' if any line matches
> A Match Strategy
> - Starts With
> - Ends With
> - Contains
> - Equals
> - Matches Regular Expression
> - Contains Regular Expression
> And then user-defined properties that indicate what to search each line of 
> text for.
> So to find lines that begin with the < character
> You would simply add a property named "Begins with Less Than" and set the 
> value to : <
> Then set the Match Strategy to Starts With
> And Routing Strategy to "Route each line to matching Property Name"
> Then, any line that begins with < will be routed to the Begins with Less Than 
> relationship.
> This would be a simple way to pull out any particular lines of interest in a 
> text file.
> I can see this being very useful for processing log files, CSV, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (NIFI-447) Add "Replacement Strategy" property to ReplaceText

[Joseph Percivall (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Percivall reassigned NIFI-447:
-

Assignee: Joseph Percivall

> Add "Replacement Strategy" property to ReplaceText
> --
>
> Key: NIFI-447
> URL: https://issues.apache.org/jira/browse/NIFI-447
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Joseph Percivall
> Fix For: 0.4.0
>
> Attachments: 
> 0001-NIFI-447-Fixed-some-typos-and-clarified-some-documen.patch, 
> NIFI-447.patch
>
>
> "Regular Expression" property should have displayName changed to "Search 
> Value"
> ReplaceText should allow for a "Replacement Strategy" with values:
> Prepend (Search Value is ignored; replacement value prepended to FlowFile)
> Append (Search Value is ignored; replacement value appended to FlowFIle)
> Regex Replace (current behavior, so this should be default)
> Linewise Regex Replace (like current behavior if Evaluation Mode is set to 
> Line-by-Line)
> Literal Replace (do not interpret the Replacement value
> The Line-by-Line property should be removed; this is breaking change!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-1027) Treat label and node of a provenance graph event as one unit

[Aldrin Piri (JIRA)]

Aldrin Piri created NIFI-1027:
-

 Summary: Treat label and node of a provenance graph event as one 
unit
 Key: NIFI-1027
 URL: https://issues.apache.org/jira/browse/NIFI-1027
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core UI
Affects Versions: 0.3.0
Reporter: Aldrin Piri
Priority: Minor


Interactions with provenance graph events can be unintuitive for user 
interaction. If navigating a graph, the labels on a provenance event node 
(JOIN, DROP, etc) cause a cursor change when hovering over the text label for 
that node but this does not allow any interaction with the event.  In contrast, 
hovering over the actual node itself, there is no cursor indication but is 
where the actions must occur.

It would be more intuitive if the node and its label were one unit, the 
entirety of which was actionable for interaction with the event.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-1029) Need to add a new attriibute to ExecuteSQL to indicate how many rows were selected

[Scott (JIRA)]

Scott created NIFI-1029:
---

 Summary: Need to add a new attriibute to ExecuteSQL to indicate 
how many rows were selected
 Key: NIFI-1029
 URL: https://issues.apache.org/jira/browse/NIFI-1029
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Extensions
Affects Versions: 0.3.0
Reporter: Scott
Priority: Critical


When running a select we need to provide a rows.count attribute so we can 
detect if there is any data. This will prevent passing no data to the 
ConvertAvroToJson and getting an error because there is no data.

Relates too NIFI-972, but I cannot figure out how to add that in.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-972) ExecuteSQL bug in createSchema() create Arvo Schema method

[Scott (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949036#comment-14949036
 ] 

Scott commented on NIFI-972:


I like the rows.count idea, I think that is the best solution, relatively quick 
to implement too!

> ExecuteSQL bug in createSchema() create Arvo Schema method
> --
>
> Key: NIFI-972
> URL: https://issues.apache.org/jira/browse/NIFI-972
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Toivo Adams
>Assignee: Toivo Adams
> Attachments: nifi-972_01oct2015.patch
>
>
> Jonathan Lyons reported:
> Just getting started with NiFi here. I am attempting to run a static query
> in MySQL using the ExecuteSQL processor. It is set to run on a 5 second
> interval. Since ExecuteSQL appears to need an input flow file I'm using a
> GenerateFlowFile processor to produce a random file every 5 seconds.
> Unfortunately, I'm getting a very vague ArrayIndexOutOfBounds exception
> when I hit play on the flow:
> java.lang.ArrayIndexOutOfBoundsException: 8
> at org.apache.avro.generic.GenericData$Record.put(GenericData.java:129)
> at org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream
> at
> org.apache.nifi.processors.standard.ExecuteSQL$1.process(ExecuteSQL.java:141)
> It’s seem to be bug in createSchema() which creates Arvo schema.
> When createSchema() will encounter unknown type, no Arvo column info will be 
> created. And Arvo schema has less columns than ResultSet row…



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-1028) Document FlowFiles and the repos in depth

[Joseph Percivall (JIRA)]

Joseph Percivall created NIFI-1028:
--

 Summary: Document FlowFiles and the repos in depth
 Key: NIFI-1028
 URL: https://issues.apache.org/jira/browse/NIFI-1028
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Documentation & Website
Reporter: Joseph Percivall
Assignee: Joseph Percivall


Currently the Documentation gives an overview of the FlowFiles and repos but 
users need a good in depth look into what a FlowFile is and how it changes when 
it goes through a flow, and there needs to be in depth documentation on the 
three repositories. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-1030) InvokeHttp basic auth password property is not marked as sensitive

[Joseph Percivall (JIRA)]

Joseph Percivall created NIFI-1030:
--

 Summary: InvokeHttp basic auth password property is not marked as 
sensitive
 Key: NIFI-1030
 URL: https://issues.apache.org/jira/browse/NIFI-1030
 Project: Apache NiFi
  Issue Type: Bug
Reporter: Joseph Percivall
Assignee: Joseph Percivall
Priority: Trivial
 Fix For: 0.4.0


The InvokeHttp allows Basic Authentication which includes a username and 
password. The property for the password is not marked as sensitive, it should 
be.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-1031) Reporting Tasks cause error messages in bootstrap.log

[Michael Moser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-1031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Moser updated NIFI-1031:

Attachment: 0001-NIFI-1031-resolve-reporting-task-error-in-flow.xml.patch

Fixed FlowConfiguration.xsd

> Reporting Tasks cause error messages in bootstrap.log
> -
>
> Key: NIFI-1031
> URL: https://issues.apache.org/jira/browse/NIFI-1031
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 0.3.0
>Reporter: Michael Moser
>Assignee: Michael Moser
>Priority: Trivial
> Fix For: 0.4.0
>
> Attachments: 
> 0001-NIFI-1031-resolve-reporting-task-error-in-flow.xml.patch
>
>
> Add any Reporting Task to NiFi configuration, restart NiFi, when restart is 
> complete observed the following error In bootstrap.log
> ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :23: 
> cvc-complex-type.2.4.a: Invalid content was found starting with element 
> 'scheduledState'. One of '{schedulingState}' is expected.
> It appears that 
> nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardFlowSerializer#addReportingTask()
>  inserts 'scheduledState' into flow.xml but
> nifi-framework-core/src/main/resources/FlowConfiguration.xsd for a 
>  uses .



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-786) Add other supporting options for configuring credentials for AWS processors

[Michael Kobit (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949828#comment-14949828
 ] 

Michael Kobit commented on NIFI-786:


[~markap14] Hey Mark, my initial idea was that the ControllerService}} would 
return an 
[AWSCredentialsProvider|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/AWSCredentialsProvider.html]
 not an 
[|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/AWSCredentials.html].
 The {{*Provider}} can be used for supplying credentials that don't change, the 
static access and secret key use case now, but also allows for complicated 
implementations, like the ones that integrate with key management systems (like 
via [STS|http://docs.aws.amazon.com/STS/latest/APIReference/Welcome.html]). I 
am fairly certain it is supported by all of the SDK vended 
{{Amazon*ServiceClient}}

For the use case of typing them in the UI, it would be supported by the, what I 
called above, {{StaticAwsCredentialsProviderService}} {{ControllerService}}. 
Users would type in the access/secret keys like they can do now, although they 
would have to set it up at the controller service level rather than at a 
processor level. This way it could also be reused across multiple AWS 
processors.

For the properties file use case, it would be supported by the, what I called 
above, {{PropertiesFileAwsCredentialsProviderService}} {{ControllerService}}. 
This implementation I would see using the AWS SDK 
[PropertiesFileCredentialsProvider|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/PropertiesFileCredentialsProvider.html].
 It covers the file based use case, but it also (currently) allows for 
credential rotation in the file since it gets loaded every time.

One of the main use cases I was thinking of is when running on AWS and you are 
["Using IAM Roles to Grant Access to AWS Resources on Amazon 
EC2"|http://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/java-dg-roles.html].
 Basically distributing the credentials to the EC2 instances themselves rather 
than managing them in your application. The keys are temporary and AWS 
automatically rotates them, so using the AWS Java SDK 
{{AWSCredentialsProvider}} implementation is preferred. 

A couple other possible use cases that could be supported fairly easily:
1. environment variables
2. Java system properties

And then there is the option of allowing consumers to define their own 
{{ControllerService}} for retrieving credentials.

> Add other supporting options for configuring credentials for AWS processors
> ---
>
> Key: NIFI-786
> URL: https://issues.apache.org/jira/browse/NIFI-786
> Project: Apache NiFi
>  Issue Type: Improvement
>Affects Versions: 0.3.0
>Reporter: Michael Kobit
>Priority: Minor
>
> I was looking at https://issues.apache.org/jira/browse/NIFI-770 and looked at 
> how the AWS processors credentials are currently configured. As a NFM you 
> have a few options with the properties right now:
> 1) set basic, static credentials
> 2) set a credentials properties filepath
> 3) set neither, use anonymous credentials
> I think it would be better if each AWS could rely on a ControllerService that 
> returns `AWSCredentialsProvider` (instead of  `AWSCredentials`) that gives 
> all of the possible implementations that could be used, rather than relying 
> on a static credentials. *Provider implementations can be refreshed and can 
> also  other more complicated implementations, but already have built in 
> support for the Static and Properties file that are provided by NiFi today.
> My thinking is that the controller service would be something like
> public interface AwsCredentialsProviderService extends ControllerService {
>   AWSCredentialsProvider getCredentialsProvider();
> }
> and you could have `StaticAwsCredentialsProviderService`, 
> `PropertiesFileAwsCredentialsProviderService`, and 
> `AnonymousAwsCredentialsProviderService` to provide the functionality that is 
> supported right now. Additional credential providers could be added later, as 
> there a bunch more AWS provided versions that I think could fit in well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (NIFI-786) Add other supporting options for configuring credentials for AWS processors

[Michael Kobit (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949828#comment-14949828
 ] 

Michael Kobit edited comment on NIFI-786 at 10/9/15 3:21 AM:
-

[~markap14] Hey Mark, my initial idea was that the {{ControllerService}} would 
return an 
[{{AWSCredentialsProvider}}|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/AWSCredentialsProvider.html]
 not an 
[{{AWSCredentials}}|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/AWSCredentials.html].
 The {{*Provider}} can be used for supplying credentials that don't change, the 
static access and secret key use case now, but also allows for complicated 
implementations, like the ones that integrate with key management systems (like 
via [STS|http://docs.aws.amazon.com/STS/latest/APIReference/Welcome.html]). I 
am fairly certain it is supported by all of the SDK vended 
{{Amazon*ServiceClient}}

For the use case of typing them in the UI, it would be supported by the, what I 
called above, {{StaticAwsCredentialsProviderService}} {{ControllerService}}. 
Users would type in the access/secret keys like they can do now, although they 
would have to set it up at the controller service level rather than at a 
processor level. This way it could also be reused across multiple AWS 
processors.

For the properties file use case, it would be supported by the, what I called 
above, {{PropertiesFileAwsCredentialsProviderService}} {{ControllerService}}. 
This implementation I would see using the AWS SDK 
[PropertiesFileCredentialsProvider|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/PropertiesFileCredentialsProvider.html].
 It covers the file based use case, but it also (currently) allows for 
credential rotation in the file since it gets loaded every time.

One of the main use cases I was thinking of is when running on AWS and you are 
["Using IAM Roles to Grant Access to AWS Resources on Amazon 
EC2"|http://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/java-dg-roles.html].
 Basically distributing the credentials to the EC2 instances themselves rather 
than managing them in your application. The keys are temporary and AWS 
automatically rotates them, so using the AWS Java SDK 
{{AWSCredentialsProvider}} implementation is preferred. 

A couple other possible use cases that could be supported fairly easily:
1. environment variables
2. Java system properties

And then there is the option of allowing consumers to define their own 
{{ControllerService}} for retrieving credentials.


was (Author: mkobit):
[~markap14] Hey Mark, my initial idea was that the ControllerService}} would 
return an 
[{{AWSCredentialsProvider}}|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/AWSCredentialsProvider.html]
 not an 
[|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/AWSCredentials.html].
 The {{*Provider}} can be used for supplying credentials that don't change, the 
static access and secret key use case now, but also allows for complicated 
implementations, like the ones that integrate with key management systems (like 
via [STS|http://docs.aws.amazon.com/STS/latest/APIReference/Welcome.html]). I 
am fairly certain it is supported by all of the SDK vended 
{{Amazon*ServiceClient}}

For the use case of typing them in the UI, it would be supported by the, what I 
called above, {{StaticAwsCredentialsProviderService}} {{ControllerService}}. 
Users would type in the access/secret keys like they can do now, although they 
would have to set it up at the controller service level rather than at a 
processor level. This way it could also be reused across multiple AWS 
processors.

For the properties file use case, it would be supported by the, what I called 
above, {{PropertiesFileAwsCredentialsProviderService}} {{ControllerService}}. 
This implementation I would see using the AWS SDK 
[PropertiesFileCredentialsProvider|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/PropertiesFileCredentialsProvider.html].
 It covers the file based use case, but it also (currently) allows for 
credential rotation in the file since it gets loaded every time.

One of the main use cases I was thinking of is when running on AWS and you are 
["Using IAM Roles to Grant Access to AWS Resources on Amazon 
EC2"|http://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/java-dg-roles.html].
 Basically distributing the credentials to the EC2 instances themselves rather 
than managing them in your application. The keys are temporary and AWS 
automatically rotates them, so using the AWS Java SDK 
{{AWSCredentialsProvider}} implementation is preferred. 

A couple other possible use cases that could be supported fairly easily:
1. environment variables
2. Java system properties

And then there is the option of allowing consumers to define their own 

[jira] [Comment Edited] (NIFI-786) Add other supporting options for configuring credentials for AWS processors

[Michael Kobit (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949828#comment-14949828
 ] 

Michael Kobit edited comment on NIFI-786 at 10/9/15 3:20 AM:
-

[~markap14] Hey Mark, my initial idea was that the ControllerService}} would 
return an 
[{{AWSCredentialsProvider}}|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/AWSCredentialsProvider.html]
 not an 
[|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/AWSCredentials.html].
 The {{*Provider}} can be used for supplying credentials that don't change, the 
static access and secret key use case now, but also allows for complicated 
implementations, like the ones that integrate with key management systems (like 
via [STS|http://docs.aws.amazon.com/STS/latest/APIReference/Welcome.html]). I 
am fairly certain it is supported by all of the SDK vended 
{{Amazon*ServiceClient}}

For the use case of typing them in the UI, it would be supported by the, what I 
called above, {{StaticAwsCredentialsProviderService}} {{ControllerService}}. 
Users would type in the access/secret keys like they can do now, although they 
would have to set it up at the controller service level rather than at a 
processor level. This way it could also be reused across multiple AWS 
processors.

For the properties file use case, it would be supported by the, what I called 
above, {{PropertiesFileAwsCredentialsProviderService}} {{ControllerService}}. 
This implementation I would see using the AWS SDK 
[PropertiesFileCredentialsProvider|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/PropertiesFileCredentialsProvider.html].
 It covers the file based use case, but it also (currently) allows for 
credential rotation in the file since it gets loaded every time.

One of the main use cases I was thinking of is when running on AWS and you are 
["Using IAM Roles to Grant Access to AWS Resources on Amazon 
EC2"|http://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/java-dg-roles.html].
 Basically distributing the credentials to the EC2 instances themselves rather 
than managing them in your application. The keys are temporary and AWS 
automatically rotates them, so using the AWS Java SDK 
{{AWSCredentialsProvider}} implementation is preferred. 

A couple other possible use cases that could be supported fairly easily:
1. environment variables
2. Java system properties

And then there is the option of allowing consumers to define their own 
{{ControllerService}} for retrieving credentials.


was (Author: mkobit):
[~markap14] Hey Mark, my initial idea was that the ControllerService}} would 
return an 
[AWSCredentialsProvider|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/AWSCredentialsProvider.html]
 not an 
[|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/AWSCredentials.html].
 The {{*Provider}} can be used for supplying credentials that don't change, the 
static access and secret key use case now, but also allows for complicated 
implementations, like the ones that integrate with key management systems (like 
via [STS|http://docs.aws.amazon.com/STS/latest/APIReference/Welcome.html]). I 
am fairly certain it is supported by all of the SDK vended 
{{Amazon*ServiceClient}}

For the use case of typing them in the UI, it would be supported by the, what I 
called above, {{StaticAwsCredentialsProviderService}} {{ControllerService}}. 
Users would type in the access/secret keys like they can do now, although they 
would have to set it up at the controller service level rather than at a 
processor level. This way it could also be reused across multiple AWS 
processors.

For the properties file use case, it would be supported by the, what I called 
above, {{PropertiesFileAwsCredentialsProviderService}} {{ControllerService}}. 
This implementation I would see using the AWS SDK 
[PropertiesFileCredentialsProvider|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/PropertiesFileCredentialsProvider.html].
 It covers the file based use case, but it also (currently) allows for 
credential rotation in the file since it gets loaded every time.

One of the main use cases I was thinking of is when running on AWS and you are 
["Using IAM Roles to Grant Access to AWS Resources on Amazon 
EC2"|http://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/java-dg-roles.html].
 Basically distributing the credentials to the EC2 instances themselves rather 
than managing them in your application. The keys are temporary and AWS 
automatically rotates them, so using the AWS Java SDK 
{{AWSCredentialsProvider}} implementation is preferred. 

A couple other possible use cases that could be supported fairly easily:
1. environment variables
2. Java system properties

And then there is the option of allowing consumers to define their own 
{{ControllerService}} 

[jira] [Updated] (NIFI-1010) When using MSSqlServer Driver with ExecuteSQL always recieve Avro parse error: Empty: Name

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-1010:
--
Assignee: Mark Payne

> When using MSSqlServer Driver with ExecuteSQL always recieve Avro parse 
> error: Empty: Name
> --
>
> Key: NIFI-1010
> URL: https://issues.apache.org/jira/browse/NIFI-1010
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 0.3.0
> Environment: MSSqlServer
>Reporter: Scott
>Assignee: Mark Payne
>  Labels: ExecuteSQL, MSSql
> Fix For: 0.4.0
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> When using ExecuteSQL with the MSSqlServerDriver from Microsoft you always 
> get a SchemParseException , Empty: Name.
> This is located in / nifi-nar-bundles / nifi-standard-bundle / 
> nifi-standard-processors / src / main / java / org / apache / nifi / 
> processors / standard / util / JdbcCommon.java line 98.
> The line should read: final String tableName = meta.getTableName(1) == "" ? 
> "table" : meta.getTableName(1);



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-1010) When using MSSqlServer Driver with ExecuteSQL always recieve Avro parse error: Empty: Name

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-1010:
--
Fix Version/s: 0.4.0

> When using MSSqlServer Driver with ExecuteSQL always recieve Avro parse 
> error: Empty: Name
> --
>
> Key: NIFI-1010
> URL: https://issues.apache.org/jira/browse/NIFI-1010
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 0.3.0
> Environment: MSSqlServer
>Reporter: Scott
>  Labels: ExecuteSQL, MSSql
> Fix For: 0.4.0
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> When using ExecuteSQL with the MSSqlServerDriver from Microsoft you always 
> get a SchemParseException , Empty: Name.
> This is located in / nifi-nar-bundles / nifi-standard-bundle / 
> nifi-standard-processors / src / main / java / org / apache / nifi / 
> processors / standard / util / JdbcCommon.java line 98.
> The line should read: final String tableName = meta.getTableName(1) == "" ? 
> "table" : meta.getTableName(1);



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-1030) InvokeHttp basic auth password property is not marked as sensitive

[Joseph Percivall (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Percivall updated NIFI-1030:
---
Attachment: NIFI-1030.patch

Added the line to make it sensitive.

> InvokeHttp basic auth password property is not marked as sensitive
> --
>
> Key: NIFI-1030
> URL: https://issues.apache.org/jira/browse/NIFI-1030
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joseph Percivall
>Assignee: Joseph Percivall
>Priority: Trivial
> Fix For: 0.4.0
>
> Attachments: NIFI-1030.patch
>
>
> The InvokeHttp allows Basic Authentication which includes a username and 
> password. The property for the password is not marked as sensitive, it should 
> be.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

nifi git commit: NIFI-1030 made InvokeHttp basic auth password a sensitive value

[aldrin]

Repository: nifi
Updated Branches:
  refs/heads/master 2583d7869 -> b4bfcc1f2


NIFI-1030 made InvokeHttp basic auth password a sensitive value

Signed-off-by: Aldrin Piri 


Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/b4bfcc1f
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/b4bfcc1f
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/b4bfcc1f

Branch: refs/heads/master
Commit: b4bfcc1f21fed3209bf4a8f187616cdbb3d1a5c9
Parents: 2583d78
Author: Joseph Percivall 
Authored: Thu Oct 8 15:33:16 2015 -0400
Committer: Aldrin Piri 
Committed: Thu Oct 8 15:46:12 2015 -0400

--
 .../main/java/org/apache/nifi/processors/standard/InvokeHTTP.java   | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/nifi/blob/b4bfcc1f/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/InvokeHTTP.java
--
diff --git 
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/InvokeHTTP.java
 
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/InvokeHTTP.java
index f16eb9c..bf1fb4e 100644
--- 
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/InvokeHTTP.java
+++ 
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/InvokeHTTP.java
@@ -277,6 +277,7 @@ public final class InvokeHTTP extends AbstractProcessor {
 .displayName("Basic Authentication Password")
 .description("The password to be used by the client to 
authenticate against the Remote URL.")
 .required(false)
+.sensitive(true)
 
.addValidator(StandardValidators.createRegexMatchingValidator(Pattern.compile("^[\\x20-\\x7e\\x80-\\xff]+$")))
 .build();
 

[jira] [Commented] (NIFI-1030) InvokeHttp basic auth password property is not marked as sensitive

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949265#comment-14949265
 ] 

ASF subversion and git services commented on NIFI-1030:
---

Commit b4bfcc1f21fed3209bf4a8f187616cdbb3d1a5c9 in nifi's branch 
refs/heads/master from [~JPercivall]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=b4bfcc1 ]

NIFI-1030 made InvokeHttp basic auth password a sensitive value

Signed-off-by: Aldrin Piri 


> InvokeHttp basic auth password property is not marked as sensitive
> --
>
> Key: NIFI-1030
> URL: https://issues.apache.org/jira/browse/NIFI-1030
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joseph Percivall
>Assignee: Joseph Percivall
>Priority: Trivial
> Fix For: 0.4.0
>
> Attachments: NIFI-1030.patch
>
>
> The InvokeHttp allows Basic Authentication which includes a username and 
> password. The property for the password is not marked as sensitive, it should 
> be.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-1030) InvokeHttp basic auth password property is not marked as sensitive

[Aldrin Piri (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aldrin Piri updated NIFI-1030:
--
Affects Version/s: 0.3.0

> InvokeHttp basic auth password property is not marked as sensitive
> --
>
> Key: NIFI-1030
> URL: https://issues.apache.org/jira/browse/NIFI-1030
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 0.3.0
>Reporter: Joseph Percivall
>Assignee: Joseph Percivall
>Priority: Trivial
> Fix For: 0.4.0
>
> Attachments: NIFI-1030.patch
>
>
> The InvokeHttp allows Basic Authentication which includes a username and 
> password. The property for the password is not marked as sensitive, it should 
> be.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1030) InvokeHttp basic auth password property is not marked as sensitive

[Aldrin Piri (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949266#comment-14949266
 ] 

Aldrin Piri commented on NIFI-1030:
---

Build and tests good, committed and pushed.  Thanks!

> InvokeHttp basic auth password property is not marked as sensitive
> --
>
> Key: NIFI-1030
> URL: https://issues.apache.org/jira/browse/NIFI-1030
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 0.3.0
>Reporter: Joseph Percivall
>Assignee: Joseph Percivall
>Priority: Trivial
> Fix For: 0.4.0
>
> Attachments: NIFI-1030.patch
>
>
> The InvokeHttp allows Basic Authentication which includes a username and 
> password. The property for the password is not marked as sensitive, it should 
> be.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-447) Add "Replacement Strategy" property to ReplaceText

[Mark Payne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Payne updated NIFI-447:

Attachment: (was: 
0001-NIFI-447-Fixed-some-typos-and-clarified-some-documen.patch)

> Add "Replacement Strategy" property to ReplaceText
> --
>
> Key: NIFI-447
> URL: https://issues.apache.org/jira/browse/NIFI-447
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Joseph Percivall
> Fix For: 0.4.0
>
> Attachments: NIFI-447.patch
>
>
> "Regular Expression" property should have displayName changed to "Search 
> Value"
> ReplaceText should allow for a "Replacement Strategy" with values:
> Prepend (Search Value is ignored; replacement value prepended to FlowFile)
> Append (Search Value is ignored; replacement value appended to FlowFIle)
> Regex Replace (current behavior, so this should be default)
> Linewise Regex Replace (like current behavior if Evaluation Mode is set to 
> Line-by-Line)
> Literal Replace (do not interpret the Replacement value
> The Line-by-Line property should be removed; this is breaking change!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-447) Add "Replacement Strategy" property to ReplaceText

[Mark Payne (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949283#comment-14949283
 ] 

Mark Payne commented on NIFI-447:
-

I replaced the existing patch because i found that ReplaceText was creating a 
very large buffer for each incoming FlowFile, which was causing some 
performance problems. This has been addressed here.

Thanks
-Mark

> Add "Replacement Strategy" property to ReplaceText
> --
>
> Key: NIFI-447
> URL: https://issues.apache.org/jira/browse/NIFI-447
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Joseph Percivall
> Fix For: 0.4.0
>
> Attachments: 
> 0001-NIFI-447-Fixed-some-typos-and-clarified-some-documen.patch, 
> NIFI-447.patch
>
>
> "Regular Expression" property should have displayName changed to "Search 
> Value"
> ReplaceText should allow for a "Replacement Strategy" with values:
> Prepend (Search Value is ignored; replacement value prepended to FlowFile)
> Append (Search Value is ignored; replacement value appended to FlowFIle)
> Regex Replace (current behavior, so this should be default)
> Linewise Regex Replace (like current behavior if Evaluation Mode is set to 
> Line-by-Line)
> Literal Replace (do not interpret the Replacement value
> The Line-by-Line property should be removed; this is breaking change!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-447) Add "Replacement Strategy" property to ReplaceText

[Mark Payne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Payne updated NIFI-447:

Attachment: 0001-NIFI-447-Fixed-some-typos-and-clarified-some-documen.patch

> Add "Replacement Strategy" property to ReplaceText
> --
>
> Key: NIFI-447
> URL: https://issues.apache.org/jira/browse/NIFI-447
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Joseph Percivall
> Fix For: 0.4.0
>
> Attachments: 
> 0001-NIFI-447-Fixed-some-typos-and-clarified-some-documen.patch, 
> NIFI-447.patch
>
>
> "Regular Expression" property should have displayName changed to "Search 
> Value"
> ReplaceText should allow for a "Replacement Strategy" with values:
> Prepend (Search Value is ignored; replacement value prepended to FlowFile)
> Append (Search Value is ignored; replacement value appended to FlowFIle)
> Regex Replace (current behavior, so this should be default)
> Linewise Regex Replace (like current behavior if Evaluation Mode is set to 
> Line-by-Line)
> Literal Replace (do not interpret the Replacement value
> The Line-by-Line property should be removed; this is breaking change!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)