[jira] [Commented] (NIFI-5479) Upgrade version of Jetty
[ https://issues.apache.org/jira/browse/NIFI-5479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711040#comment-16711040 ] ASF GitHub Bot commented on NIFI-5479: -- Github user rahulbhanushali commented on the issue: https://github.com/apache/nifi/pull/3034 Yes, the reason being I still this logs on 1.80. and the node doesn't connect to the cluster. There's no other log apart from this in the file so I am assuming this may be the reason the node doesn't connect to the cluster. > Upgrade version of Jetty > > > Key: NIFI-5479 > URL: https://issues.apache.org/jira/browse/NIFI-5479 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.7.1 >Reporter: Andy LoPresto >Assignee: Matt Gilman >Priority: Blocker > Fix For: 1.8.0 > > > Upgrade to a new version of Jetty. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #3034: NIFI-5479 - Fixed up dependencies to remove the WARNs caus...
Github user rahulbhanushali commented on the issue: https://github.com/apache/nifi/pull/3034 Yes, the reason being I still this logs on 1.80. and the node doesn't connect to the cluster. There's no other log apart from this in the file so I am assuming this may be the reason the node doesn't connect to the cluster. ---
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710968#comment-16710968 ] ASF GitHub Bot commented on NIFI-4621: -- Github user ijokarumawak commented on a diff in the pull request: https://github.com/apache/nifi/pull/3201#discussion_r239324396 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ListFTP.java --- @@ -35,17 +36,21 @@ import org.apache.nifi.components.ValidationResult; import org.apache.nifi.components.state.Scope; import org.apache.nifi.context.PropertyContext; +import org.apache.nifi.expression.ExpressionLanguageScope; import org.apache.nifi.processor.ProcessContext; import org.apache.nifi.processor.util.list.ListedEntityTracker; import org.apache.nifi.processors.standard.util.FileTransfer; import org.apache.nifi.processors.standard.util.FTPTransfer; @PrimaryNodeOnly @TriggerSerially -@InputRequirement(Requirement.INPUT_FORBIDDEN) +@InputRequirement(Requirement.INPUT_ALLOWED) --- End diff -- Let's design how we improve this processors (or create other processors if necessary) first. I remember there was some discussion on JIRA or ML before. Please check those, too. If PATh can be supplied by incoming FlowFiles, then I personally prefer passing minimum timestamps by FlowFiles, too. > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Peter Wicks >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #3201: NIFI-4621
Github user ijokarumawak commented on a diff in the pull request: https://github.com/apache/nifi/pull/3201#discussion_r239324396 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ListFTP.java --- @@ -35,17 +36,21 @@ import org.apache.nifi.components.ValidationResult; import org.apache.nifi.components.state.Scope; import org.apache.nifi.context.PropertyContext; +import org.apache.nifi.expression.ExpressionLanguageScope; import org.apache.nifi.processor.ProcessContext; import org.apache.nifi.processor.util.list.ListedEntityTracker; import org.apache.nifi.processors.standard.util.FileTransfer; import org.apache.nifi.processors.standard.util.FTPTransfer; @PrimaryNodeOnly @TriggerSerially -@InputRequirement(Requirement.INPUT_FORBIDDEN) +@InputRequirement(Requirement.INPUT_ALLOWED) --- End diff -- Let's design how we improve this processors (or create other processors if necessary) first. I remember there was some discussion on JIRA or ML before. Please check those, too. If PATh can be supplied by incoming FlowFiles, then I personally prefer passing minimum timestamps by FlowFiles, too. ---
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710921#comment-16710921 ] ASF GitHub Bot commented on NIFI-4621: -- Github user kislayom commented on a diff in the pull request: https://github.com/apache/nifi/pull/3201#discussion_r239314421 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ListFTP.java --- @@ -35,17 +36,21 @@ import org.apache.nifi.components.ValidationResult; import org.apache.nifi.components.state.Scope; import org.apache.nifi.context.PropertyContext; +import org.apache.nifi.expression.ExpressionLanguageScope; import org.apache.nifi.processor.ProcessContext; import org.apache.nifi.processor.util.list.ListedEntityTracker; import org.apache.nifi.processors.standard.util.FileTransfer; import org.apache.nifi.processors.standard.util.FTPTransfer; @PrimaryNodeOnly @TriggerSerially -@InputRequirement(Requirement.INPUT_FORBIDDEN) +@InputRequirement(Requirement.INPUT_ALLOWED) --- End diff -- Sure, based on the input provided i will try to identify required changes. The latest timestamp management part might be tricky are we required to keep latest timestamp per path, hostname, and user wise or just the latest one ? I have so far assumed to manage latest timestamp only for the latest. If we plan to manages all timestamps i will need to rework on a number of things here, and may would like to approach the problem starting with proposed design. > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Peter Wicks >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[
https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710923#comment-16710923
]
ASF GitHub Bot commented on NIFI-4621:
--
Github user kislayom commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3201#discussion_r239314794
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ListFTP.java
---
@@ -35,17 +36,21 @@
import org.apache.nifi.components.ValidationResult;
import org.apache.nifi.components.state.Scope;
import org.apache.nifi.context.PropertyContext;
+import org.apache.nifi.expression.ExpressionLanguageScope;
import org.apache.nifi.processor.ProcessContext;
import org.apache.nifi.processor.util.list.ListedEntityTracker;
import org.apache.nifi.processors.standard.util.FileTransfer;
import org.apache.nifi.processors.standard.util.FTPTransfer;
@PrimaryNodeOnly
@TriggerSerially
-@InputRequirement(Requirement.INPUT_FORBIDDEN)
+@InputRequirement(Requirement.INPUT_ALLOWED)
@Tags({"list", "ftp", "remote", "ingest", "source", "input", "files"})
@CapabilityDescription("Performs a listing of the files residing on an FTP
server. For each file that is found on the remote server, a new FlowFile will
be created with the filename attribute "
+ "set to the name of the file on the remote server. This can then be
used in conjunction with FetchFTP in order to fetch those files.")
+@DynamicProperty(name = "Relationship Name", value = "Attribute Expression
Language",
+expressionLanguageScope =
ExpressionLanguageScope.FLOWFILE_ATTRIBUTES, description = "Routes FlowFiles
whose attributes match the "
++ "Attribute Expression Language specified in the Dynamic Property
Value to the Relationship specified in the Dynamic Property Key")
--- End diff --
It is used to set the scope of Expression language. More details as under:
https://static.javadoc.io/org.apache.nifi/nifi-api/1.7.0/org/apache/nifi/expression/ExpressionLanguageScope.html
> Indicates the scope of expression language on a property descriptor.
Scope of the expression language is hierarchical. NONE -> VARIABLE_REGISTRY ->
FLOWFILE_ATTRIBUTES When scope is set to FlowFiles attributes, variables are
evaluated against attributes of each incoming flow file. If no matching
attribute is found, variable registry will be checked. NONE - expression
language is not supported VARIABLE_REGISTRY is hierarchically constructed as
below: | Variables defined at process group level and then, recursively, up
| to the higher process group until the root process group. |--- Variables
defined in custom properties files through the |
nifi.variable.registry.properties property in nifi.properties file. |--
Environment variables defined at JVM level and system properties.
FLOWFILE_ATTRIBUTES - will check attributes of each individual flow file
>
> Allow inputs to ListSFTP and ListFTP
>
>
> Key: NIFI-4621
> URL: https://issues.apache.org/jira/browse/NIFI-4621
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.4.0
>Reporter: Soumya Shanta Ghosh
>Assignee: Peter Wicks
>Priority: Critical
>
> ListSFTP supports listing of the supplied directory (Remote Path)
> out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password"
> / "Private Key Passphrase".
> The password can change at a regular interval (depending on organization
> policy) or the Hostname or the Remote Path can change based on some other
> requirement.
> This is a case to allow ListSFTP to leverage the use of Nifi Expression
> language so that the values of Hostname, Password and/or Remote Path can be
> set based on the attributes of an incoming flow file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #3201: NIFI-4621
Github user kislayom commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3201#discussion_r239314794
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ListFTP.java
---
@@ -35,17 +36,21 @@
import org.apache.nifi.components.ValidationResult;
import org.apache.nifi.components.state.Scope;
import org.apache.nifi.context.PropertyContext;
+import org.apache.nifi.expression.ExpressionLanguageScope;
import org.apache.nifi.processor.ProcessContext;
import org.apache.nifi.processor.util.list.ListedEntityTracker;
import org.apache.nifi.processors.standard.util.FileTransfer;
import org.apache.nifi.processors.standard.util.FTPTransfer;
@PrimaryNodeOnly
@TriggerSerially
-@InputRequirement(Requirement.INPUT_FORBIDDEN)
+@InputRequirement(Requirement.INPUT_ALLOWED)
@Tags({"list", "ftp", "remote", "ingest", "source", "input", "files"})
@CapabilityDescription("Performs a listing of the files residing on an FTP
server. For each file that is found on the remote server, a new FlowFile will
be created with the filename attribute "
+ "set to the name of the file on the remote server. This can then be
used in conjunction with FetchFTP in order to fetch those files.")
+@DynamicProperty(name = "Relationship Name", value = "Attribute Expression
Language",
+expressionLanguageScope =
ExpressionLanguageScope.FLOWFILE_ATTRIBUTES, description = "Routes FlowFiles
whose attributes match the "
++ "Attribute Expression Language specified in the Dynamic Property
Value to the Relationship specified in the Dynamic Property Key")
--- End diff --
It is used to set the scope of Expression language. More details as under:
https://static.javadoc.io/org.apache.nifi/nifi-api/1.7.0/org/apache/nifi/expression/ExpressionLanguageScope.html
> Indicates the scope of expression language on a property descriptor.
Scope of the expression language is hierarchical. NONE -> VARIABLE_REGISTRY ->
FLOWFILE_ATTRIBUTES When scope is set to FlowFiles attributes, variables are
evaluated against attributes of each incoming flow file. If no matching
attribute is found, variable registry will be checked. NONE - expression
language is not supported VARIABLE_REGISTRY is hierarchically constructed as
below: | Variables defined at process group level and then, recursively, up
| to the higher process group until the root process group. |--- Variables
defined in custom properties files through the |
nifi.variable.registry.properties property in nifi.properties file. |--
Environment variables defined at JVM level and system properties.
FLOWFILE_ATTRIBUTES - will check attributes of each individual flow file
>
---
[GitHub] nifi pull request #3201: NIFI-4621
Github user kislayom commented on a diff in the pull request: https://github.com/apache/nifi/pull/3201#discussion_r239314421 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ListFTP.java --- @@ -35,17 +36,21 @@ import org.apache.nifi.components.ValidationResult; import org.apache.nifi.components.state.Scope; import org.apache.nifi.context.PropertyContext; +import org.apache.nifi.expression.ExpressionLanguageScope; import org.apache.nifi.processor.ProcessContext; import org.apache.nifi.processor.util.list.ListedEntityTracker; import org.apache.nifi.processors.standard.util.FileTransfer; import org.apache.nifi.processors.standard.util.FTPTransfer; @PrimaryNodeOnly @TriggerSerially -@InputRequirement(Requirement.INPUT_FORBIDDEN) +@InputRequirement(Requirement.INPUT_ALLOWED) --- End diff -- Sure, based on the input provided i will try to identify required changes. The latest timestamp management part might be tricky are we required to keep latest timestamp per path, hostname, and user wise or just the latest one ? I have so far assumed to manage latest timestamp only for the latest. If we plan to manages all timestamps i will need to rework on a number of things here, and may would like to approach the problem starting with proposed design. ---
[jira] [Updated] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-4621: Component/s: Extensions > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Peter Wicks >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (NIFI-5639) PublishAMQP Processor
[ https://issues.apache.org/jira/browse/NIFI-5639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710907#comment-16710907 ] Kislay Kumar edited comment on NIFI-5639 at 12/6/18 2:33 AM: - [~patricker] I have used the processor with ancient version of nifi (0.6). At the time we were working on near real time analytics project and the integration was with RabbitMq. I am not sure of its demand now, given most use cases have moved to Kafka or alike. I will try looking for other jira as well, if however if you have some interesting jira in mind please recommend. was (Author: kislayom): [~patricker] I have used the processor with ancient version of nifi (0.6). At the time we were working on near real time analytics project and the integration was with RabbitMq. I am not sure of its demand now, given most use cases have moved to Kafka or alike. > PublishAMQP Processor > --- > > Key: NIFI-5639 > URL: https://issues.apache.org/jira/browse/NIFI-5639 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.5.0, 1.6.0, 1.7.0, 1.7.1 >Reporter: Mohammed Nadeem >Priority: Critical > Original Estimate: 96h > Remaining Estimate: 96h > > *PublishAMQP* Processor is routing incoming flowfile to success relationship > though invalid properties are configured. The processor is not throwing any > errors when invalid properties are configured at run-time. When the message > is not published with invalid properties, the processor silently routes the > message to success when it is suppose to route to failure relationship > indicating message was not published to the queue. This is a bug and no > proper error handling when failures occur *(nifi-amqp-bundle)* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[
https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710903#comment-16710903
]
Kislay Kumar commented on NIFI-4621:
[~patricker] Some code snippets from SFTPTransfer.java is under:
*Username, hostname, port:*
{quote}final String username =
ctx.getProperty(USERNAME).evaluateAttributeExpressions(flowFile).getValue();
final Session session = jsch.getSession(username,
ctx.getProperty(HOSTNAME).evaluateAttributeExpressions(flowFile).getValue(),
ctx.getProperty(PORT).evaluateAttributeExpressions(flowFile).asInteger().intValue());
{quote}
*privateKeyFile, password*
{quote}
final String privateKeyFile =
ctx.getProperty(PRIVATE_KEY_PATH).evaluateAttributeExpressions(flowFile).getValue();
if (privateKeyFile != null) {
jsch.addIdentity(privateKeyFile,
ctx.getProperty(PRIVATE_KEY_PASSPHRASE).evaluateAttributeExpressions(flowFile).getValue());
}
final String password =
ctx.getProperty(FileTransfer.PASSWORD).evaluateAttributeExpressions(flowFile).getValue();
{quote}
> Allow inputs to ListSFTP and ListFTP
>
>
> Key: NIFI-4621
> URL: https://issues.apache.org/jira/browse/NIFI-4621
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.4.0
>Reporter: Soumya Shanta Ghosh
>Assignee: Peter Wicks
>Priority: Critical
>
> ListSFTP supports listing of the supplied directory (Remote Path)
> out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password"
> / "Private Key Passphrase".
> The password can change at a regular interval (depending on organization
> policy) or the Hostname or the Remote Path can change based on some other
> requirement.
> This is a case to allow ListSFTP to leverage the use of Nifi Expression
> language so that the values of Hostname, Password and/or Remote Path can be
> set based on the attributes of an incoming flow file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5639) PublishAMQP Processor
[ https://issues.apache.org/jira/browse/NIFI-5639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710907#comment-16710907 ] Kislay Kumar commented on NIFI-5639: [~patricker] I have used the processor with ancient version of nifi (0.6). At the time we were working on near real time analytics project and the integration was with RabbitMq. I am not sure of its demand now, given most use cases have moved to Kafka or alike. > PublishAMQP Processor > --- > > Key: NIFI-5639 > URL: https://issues.apache.org/jira/browse/NIFI-5639 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.5.0, 1.6.0, 1.7.0, 1.7.1 >Reporter: Mohammed Nadeem >Priority: Critical > Original Estimate: 96h > Remaining Estimate: 96h > > *PublishAMQP* Processor is routing incoming flowfile to success relationship > though invalid properties are configured. The processor is not throwing any > errors when invalid properties are configured at run-time. When the message > is not published with invalid properties, the processor silently routes the > message to success when it is suppose to route to failure relationship > indicating message was not published to the queue. This is a bug and no > proper error handling when failures occur *(nifi-amqp-bundle)* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710909#comment-16710909 ] ASF GitHub Bot commented on NIFI-4621: -- Github user ijokarumawak commented on a diff in the pull request: https://github.com/apache/nifi/pull/3201#discussion_r239310311 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ListFTP.java --- @@ -35,17 +36,21 @@ import org.apache.nifi.components.ValidationResult; import org.apache.nifi.components.state.Scope; import org.apache.nifi.context.PropertyContext; +import org.apache.nifi.expression.ExpressionLanguageScope; import org.apache.nifi.processor.ProcessContext; import org.apache.nifi.processor.util.list.ListedEntityTracker; import org.apache.nifi.processors.standard.util.FileTransfer; import org.apache.nifi.processors.standard.util.FTPTransfer; @PrimaryNodeOnly @TriggerSerially -@InputRequirement(Requirement.INPUT_FORBIDDEN) +@InputRequirement(Requirement.INPUT_ALLOWED) --- End diff -- Changing input requirement is not enough if we want to support incoming FlowFiles. Processors need to: 1. Take input FlowFiles, LogMessage processor can be the simplest example, please look [here](https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/LogMessage.java#L125) 2. Evaluate ExpressionLanguages using the FlowFiles, again LogMessage as [an example](https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/LogMessage.java#L130) Currently [FTPTransfer.getListing()](https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/FTPTransfer.java#L187) doesn't do above things. And we also need to support existing use-cases with no input connection. Also, we need to track listing state such as latest timestamps ... etc. We need to do that efficiently with dynamically changing target paths if it's configured by incoming FlowFiles attribute. There are more things need to be done. > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Peter Wicks >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #3201: NIFI-4621
Github user ijokarumawak commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3201#discussion_r239307533
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ListFTP.java
---
@@ -35,17 +36,21 @@
import org.apache.nifi.components.ValidationResult;
import org.apache.nifi.components.state.Scope;
import org.apache.nifi.context.PropertyContext;
+import org.apache.nifi.expression.ExpressionLanguageScope;
import org.apache.nifi.processor.ProcessContext;
import org.apache.nifi.processor.util.list.ListedEntityTracker;
import org.apache.nifi.processors.standard.util.FileTransfer;
import org.apache.nifi.processors.standard.util.FTPTransfer;
@PrimaryNodeOnly
@TriggerSerially
-@InputRequirement(Requirement.INPUT_FORBIDDEN)
+@InputRequirement(Requirement.INPUT_ALLOWED)
@Tags({"list", "ftp", "remote", "ingest", "source", "input", "files"})
@CapabilityDescription("Performs a listing of the files residing on an FTP
server. For each file that is found on the remote server, a new FlowFile will
be created with the filename attribute "
+ "set to the name of the file on the remote server. This can then be
used in conjunction with FetchFTP in order to fetch those files.")
+@DynamicProperty(name = "Relationship Name", value = "Attribute Expression
Language",
+expressionLanguageScope =
ExpressionLanguageScope.FLOWFILE_ATTRIBUTES, description = "Routes FlowFiles
whose attributes match the "
++ "Attribute Expression Language specified in the Dynamic Property
Value to the Relationship specified in the Dynamic Property Key")
--- End diff --
What's this used for?
---
[GitHub] nifi pull request #3201: NIFI-4621
Github user ijokarumawak commented on a diff in the pull request: https://github.com/apache/nifi/pull/3201#discussion_r239310311 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ListFTP.java --- @@ -35,17 +36,21 @@ import org.apache.nifi.components.ValidationResult; import org.apache.nifi.components.state.Scope; import org.apache.nifi.context.PropertyContext; +import org.apache.nifi.expression.ExpressionLanguageScope; import org.apache.nifi.processor.ProcessContext; import org.apache.nifi.processor.util.list.ListedEntityTracker; import org.apache.nifi.processors.standard.util.FileTransfer; import org.apache.nifi.processors.standard.util.FTPTransfer; @PrimaryNodeOnly @TriggerSerially -@InputRequirement(Requirement.INPUT_FORBIDDEN) +@InputRequirement(Requirement.INPUT_ALLOWED) --- End diff -- Changing input requirement is not enough if we want to support incoming FlowFiles. Processors need to: 1. Take input FlowFiles, LogMessage processor can be the simplest example, please look [here](https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/LogMessage.java#L125) 2. Evaluate ExpressionLanguages using the FlowFiles, again LogMessage as [an example](https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/LogMessage.java#L130) Currently [FTPTransfer.getListing()](https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/FTPTransfer.java#L187) doesn't do above things. And we also need to support existing use-cases with no input connection. Also, we need to track listing state such as latest timestamps ... etc. We need to do that efficiently with dynamically changing target paths if it's configured by incoming FlowFiles attribute. There are more things need to be done. ---
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[
https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710908#comment-16710908
]
ASF GitHub Bot commented on NIFI-4621:
--
Github user ijokarumawak commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3201#discussion_r239307533
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ListFTP.java
---
@@ -35,17 +36,21 @@
import org.apache.nifi.components.ValidationResult;
import org.apache.nifi.components.state.Scope;
import org.apache.nifi.context.PropertyContext;
+import org.apache.nifi.expression.ExpressionLanguageScope;
import org.apache.nifi.processor.ProcessContext;
import org.apache.nifi.processor.util.list.ListedEntityTracker;
import org.apache.nifi.processors.standard.util.FileTransfer;
import org.apache.nifi.processors.standard.util.FTPTransfer;
@PrimaryNodeOnly
@TriggerSerially
-@InputRequirement(Requirement.INPUT_FORBIDDEN)
+@InputRequirement(Requirement.INPUT_ALLOWED)
@Tags({"list", "ftp", "remote", "ingest", "source", "input", "files"})
@CapabilityDescription("Performs a listing of the files residing on an FTP
server. For each file that is found on the remote server, a new FlowFile will
be created with the filename attribute "
+ "set to the name of the file on the remote server. This can then be
used in conjunction with FetchFTP in order to fetch those files.")
+@DynamicProperty(name = "Relationship Name", value = "Attribute Expression
Language",
+expressionLanguageScope =
ExpressionLanguageScope.FLOWFILE_ATTRIBUTES, description = "Routes FlowFiles
whose attributes match the "
++ "Attribute Expression Language specified in the Dynamic Property
Value to the Relationship specified in the Dynamic Property Key")
--- End diff --
What's this used for?
> Allow inputs to ListSFTP and ListFTP
>
>
> Key: NIFI-4621
> URL: https://issues.apache.org/jira/browse/NIFI-4621
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.4.0
>Reporter: Soumya Shanta Ghosh
>Assignee: Peter Wicks
>Priority: Critical
>
> ListSFTP supports listing of the supplied directory (Remote Path)
> out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password"
> / "Private Key Passphrase".
> The password can change at a regular interval (depending on organization
> policy) or the Hostname or the Remote Path can change based on some other
> requirement.
> This is a case to allow ListSFTP to leverage the use of Nifi Expression
> language so that the values of Hostname, Password and/or Remote Path can be
> set based on the attributes of an incoming flow file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710839#comment-16710839 ] Kislay Kumar edited comment on NIFI-4621 at 12/6/18 2:29 AM: - Hi [~patricker] I also assumed (as the reporter mentioned) initially that we might be missing EL on the processors properties. However I ended up finding that almost all properties are already EL enabled. It was only that the processor were not allowed to have input connection. If you would like i am happy to post some evidence for that. And i did test EL on the properties mentioned in the Jira. However do you think that i may be missing on some other aspects of the jira ? was (Author: kislayom): Hi [~patricker] I also assumed (as the reporter mentioned) initially that we might be missing EL on the processors properties. However I ended up finding that almost all properties are already EL enabled. It was only that the processor were not allowed to have input connection. If you would like i am happy to post some evidence for that. And i did test EL on the properties mentioned in the Jira. > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Peter Wicks >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[
https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710903#comment-16710903
]
Kislay Kumar edited comment on NIFI-4621 at 12/6/18 2:25 AM:
-
[~patricker] Some code snippets from SFTPTransfer.java which implements
processors's functionality is under:
*Username, hostname, port:*
{quote}final String username =
ctx.getProperty(USERNAME).*evaluateAttributeExpressions*(flowFile).getValue();
final Session session = jsch.getSession(username,
ctx.getProperty(HOSTNAME).*evaluateAttributeExpressions*(flowFile).getValue(),
ctx.getProperty(PORT).*evaluateAttributeExpressions*(flowFile).asInteger().intValue());
{quote}
*privateKeyFile, password*
{quote}final String privateKeyFile =
ctx.getProperty(PRIVATE_KEY_PATH).evaluateAttributeExpressions(flowFile).getValue();
if (privateKeyFile != null)
Unknown macro: \{ jsch.addIdentity(privateKeyFile,
ctx.getProperty(PRIVATE_KEY_PASSPHRASE).*evaluateAttributeExpressions*(flowFile).getValue());
}
final String password =
ctx.getProperty(FileTransfer.PASSWORD).evaluateAttributeExpressions(flowFile).getValue();
{quote}
was (Author: kislayom):
[~patricker] Some code snippets from SFTPTransfer.java which implements
processors's functionality is under:
*Username, hostname, port:*
{quote}final String username =
ctx.getProperty(USERNAME).evaluateAttributeExpressions(flowFile).getValue();
final Session session = jsch.getSession(username,
ctx.getProperty(HOSTNAME).evaluateAttributeExpressions(flowFile).getValue(),
ctx.getProperty(PORT).evaluateAttributeExpressions(flowFile).asInteger().intValue());
{quote}
*privateKeyFile, password*
{quote}final String privateKeyFile =
ctx.getProperty(PRIVATE_KEY_PATH).evaluateAttributeExpressions(flowFile).getValue();
if (privateKeyFile != null)
Unknown macro: \{ jsch.addIdentity(privateKeyFile,
ctx.getProperty(PRIVATE_KEY_PASSPHRASE).evaluateAttributeExpressions(flowFile).getValue());
}
final String password =
ctx.getProperty(FileTransfer.PASSWORD).evaluateAttributeExpressions(flowFile).getValue();
{quote}
> Allow inputs to ListSFTP and ListFTP
>
>
> Key: NIFI-4621
> URL: https://issues.apache.org/jira/browse/NIFI-4621
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.4.0
>Reporter: Soumya Shanta Ghosh
>Assignee: Peter Wicks
>Priority: Critical
>
> ListSFTP supports listing of the supplied directory (Remote Path)
> out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password"
> / "Private Key Passphrase".
> The password can change at a regular interval (depending on organization
> policy) or the Hostname or the Remote Path can change based on some other
> requirement.
> This is a case to allow ListSFTP to leverage the use of Nifi Expression
> language so that the values of Hostname, Password and/or Remote Path can be
> set based on the attributes of an incoming flow file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[
https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710903#comment-16710903
]
Kislay Kumar edited comment on NIFI-4621 at 12/6/18 2:24 AM:
-
[~patricker] Some code snippets from SFTPTransfer.java which implements
processors's functionality is under:
*Username, hostname, port:*
{quote}final String username =
ctx.getProperty(USERNAME).evaluateAttributeExpressions(flowFile).getValue();
final Session session = jsch.getSession(username,
ctx.getProperty(HOSTNAME).evaluateAttributeExpressions(flowFile).getValue(),
ctx.getProperty(PORT).evaluateAttributeExpressions(flowFile).asInteger().intValue());
{quote}
*privateKeyFile, password*
{quote}final String privateKeyFile =
ctx.getProperty(PRIVATE_KEY_PATH).evaluateAttributeExpressions(flowFile).getValue();
if (privateKeyFile != null)
Unknown macro: \{ jsch.addIdentity(privateKeyFile,
ctx.getProperty(PRIVATE_KEY_PASSPHRASE).evaluateAttributeExpressions(flowFile).getValue());
}
final String password =
ctx.getProperty(FileTransfer.PASSWORD).evaluateAttributeExpressions(flowFile).getValue();
{quote}
was (Author: kislayom):
[~patricker] Some code snippets from SFTPTransfer.java is under:
*Username, hostname, port:*
{quote}final String username =
ctx.getProperty(USERNAME).evaluateAttributeExpressions(flowFile).getValue();
final Session session = jsch.getSession(username,
ctx.getProperty(HOSTNAME).evaluateAttributeExpressions(flowFile).getValue(),
ctx.getProperty(PORT).evaluateAttributeExpressions(flowFile).asInteger().intValue());
{quote}
*privateKeyFile, password*
{quote}
final String privateKeyFile =
ctx.getProperty(PRIVATE_KEY_PATH).evaluateAttributeExpressions(flowFile).getValue();
if (privateKeyFile != null) {
jsch.addIdentity(privateKeyFile,
ctx.getProperty(PRIVATE_KEY_PASSPHRASE).evaluateAttributeExpressions(flowFile).getValue());
}
final String password =
ctx.getProperty(FileTransfer.PASSWORD).evaluateAttributeExpressions(flowFile).getValue();
{quote}
> Allow inputs to ListSFTP and ListFTP
>
>
> Key: NIFI-4621
> URL: https://issues.apache.org/jira/browse/NIFI-4621
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.4.0
>Reporter: Soumya Shanta Ghosh
>Assignee: Peter Wicks
>Priority: Critical
>
> ListSFTP supports listing of the supplied directory (Remote Path)
> out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password"
> / "Private Key Passphrase".
> The password can change at a regular interval (depending on organization
> policy) or the Hostname or the Remote Path can change based on some other
> requirement.
> This is a case to allow ListSFTP to leverage the use of Nifi Expression
> language so that the values of Hostname, Password and/or Remote Path can be
> set based on the attributes of an incoming flow file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710839#comment-16710839 ] Kislay Kumar edited comment on NIFI-4621 at 12/6/18 2:18 AM: - Hi [~patricker] I also assumed (as the reporter mentioned) initially that we might be missing EL on the processors properties. However I ended up finding that almost all properties are already EL enabled. It was only that the processor were not allowed to have input connection. If you would like i am happy to post some evidence for that. And i did test EL on the properties mentioned in the Jira. was (Author: kislayom): Hi [~patricker] I also assumed (as the reported mentioned) initially that we might be missing EL on the processors properties. However I ended up finding that almost all properties are already EL enabled. It was only that the processor were not allowed to have input connection. If you would like i am happy to post some evidence for that. And i did test EL on the properties mentioned in the Jira. > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Peter Wicks >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710901#comment-16710901 ] Kislay Kumar commented on NIFI-4621: [~ijokarumawak] sure, I thought of putting it in review but could not find it. > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Peter Wicks >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-4621: Fix Version/s: (was: 1.9.0) > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Peter Wicks >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-5874) CSVRecordSetWriter inject transformed backslash sequences from input
[
https://issues.apache.org/jira/browse/NIFI-5874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ed Berezitsky updated NIFI-5874:
Description:
If there is backslash sequence (like \t, \n, etc) in the input,
CSVRecordSetWriter transforms them into actual characters (new line, tab, etc)
in output record.
For example, input record:
{code:java}
case,a,a1
tab,=\t=,-
{code}
Update Record with `/a1: /a` (just copy value from one field to another)
JsonRecordSetWriter will produce:
{code:java}
[{"case":"tab","a":"=\t=","a1":"=\t="}]{code}
while CSVRecordSetWriter will produce:
{code:java}
case,a,a1
tab,= =,= =
{code}
there is a actual "tab" in between "="
was:
If there is backslash sequence (like \t, \n, etc) in the input,
CSVRecordSetWriter transforms them into actual characters (new line, tab, etc)
in output record.
For example, input record:
{code:java}
case,a,a1
tab,=\t=,-
{code}
Update Record with `/a1: /a` (just copy value from one field to another)
JsonRecordSetWriter will produce:
{code:java}
[{"case":"tab","a":"=\t=","a1":"=\t="}]{code}
while CSVRecordSetWriter will produce:
{code:java}
case,a,a1
tab,= =,= =
{code}
there is a actual "tab" in between "="
> CSVRecordSetWriter inject transformed backslash sequences from input
>
>
> Key: NIFI-5874
> URL: https://issues.apache.org/jira/browse/NIFI-5874
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.8.0
>Reporter: Ed Berezitsky
>Priority: Major
>
> If there is backslash sequence (like \t, \n, etc) in the input,
> CSVRecordSetWriter transforms them into actual characters (new line, tab,
> etc) in output record.
> For example, input record:
>
> {code:java}
> case,a,a1
> tab,=\t=,-
> {code}
>
> Update Record with `/a1: /a` (just copy value from one field to another)
> JsonRecordSetWriter will produce:
> {code:java}
> [{"case":"tab","a":"=\t=","a1":"=\t="}]{code}
> while CSVRecordSetWriter will produce:
> {code:java}
> case,a,a1
> tab,= =,= =
> {code}
> there is a actual "tab" in between "="
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-5874) CSVRecordSetWriter inject transformed backslash sequences from input
[
https://issues.apache.org/jira/browse/NIFI-5874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ed Berezitsky updated NIFI-5874:
Description:
If there is backslash sequence (like \t, \n, etc) in the input,
CSVRecordSetWriter transforms them into actual characters (new line, tab, etc)
in output record.
For example, input record:
{code:java}
case,a,a1
tab,=\t=,-
{code}
Update Record with `/a1: /a` (just copy value from one field to another)
JsonRecordSetWriter will produce:
{code:java}
[{"case":"tab","a":"=\t=","a1":"=\t="}]{code}
while CSVRecordSetWriter will produce:
{code:java}
case,a,a1
tab,= =,= =
{code}
there is a actual "tab" in between "="
was:
If there is backslash sequence (like \t, \n, etc) in the input,
CSVRecordSetWriter transforms them into actual characters (new line, tab, etc)
in output record.
For example, input record:
{code:java}
case,a,a1
tab,=\t=,-
{code}
Update Record with `/a1: /a` (just copy value from one field to another)
JsonRecordSetWriter will produce:
[\\{"case":"tab","a":"=\t=","a1":"=\t="}]
while CSVRecordSetWriter will produce:
{code:java}
case,a,a1
tab,= =,= =
{code}
there is a actual "tab" in between "="
> CSVRecordSetWriter inject transformed backslash sequences from input
>
>
> Key: NIFI-5874
> URL: https://issues.apache.org/jira/browse/NIFI-5874
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.8.0
>Reporter: Ed Berezitsky
>Priority: Major
>
> If there is backslash sequence (like \t, \n, etc) in the input,
> CSVRecordSetWriter transforms them into actual characters (new line, tab,
> etc) in output record.
> For example, input record:
>
> {code:java}
> case,a,a1
> tab,=\t=,-
> {code}
>
> Update Record with `/a1: /a` (just copy value from one field to another)
> JsonRecordSetWriter will produce:
> {code:java}
> [{"case":"tab","a":"=\t=","a1":"=\t="}]{code}
> while CSVRecordSetWriter will produce:
>
> {code:java}
> case,a,a1
> tab,= =,= =
> {code}
> there is a actual "tab" in between "="
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-5874) CSVRecordSetWriter inject transformed backslash sequences from input
[
https://issues.apache.org/jira/browse/NIFI-5874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ed Berezitsky updated NIFI-5874:
Description:
If there is backslash sequence (like \t, \n, etc) in the input,
CSVRecordSetWriter transforms them into actual characters (new line, tab, etc)
in output record.
For example, input record:
{code:java}
case,a,a1
tab,=\t=,-
{code}
Update Record with `/a1: /a` (just copy value from one field to another)
JsonRecordSetWriter will produce:
[\\{"case":"tab","a":"=\t=","a1":"=\t="}]
while CSVRecordSetWriter will produce:
{code:java}
case,a,a1
tab,= =,= =
{code}
there is a actual "tab" in between "="
was:
If there is backslash sequence (like \t, \n, etc) in the input,
CSVRecordSetWriter transforms them into actual characters (new line, tab, etc)
in output record.
For example, input record:
```
case,a,a1
period,=\t=,-
```
Update Record with `/a1: /a` (just copy value from one field to another)
JsonRecordSetWriter will produce:
```
[\{"case":"period","a":"=\t=","a1":"=\t="}]
```
while CSVRecordSetWriter will produce:
```
case,a,a1
period,= =,= =
```
there is a actual "tab" in between "="
> CSVRecordSetWriter inject transformed backslash sequences from input
>
>
> Key: NIFI-5874
> URL: https://issues.apache.org/jira/browse/NIFI-5874
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.8.0
>Reporter: Ed Berezitsky
>Priority: Major
>
> If there is backslash sequence (like \t, \n, etc) in the input,
> CSVRecordSetWriter transforms them into actual characters (new line, tab,
> etc) in output record.
> For example, input record:
>
> {code:java}
> case,a,a1
> tab,=\t=,-
> {code}
>
> Update Record with `/a1: /a` (just copy value from one field to another)
> JsonRecordSetWriter will produce:
> [\\{"case":"tab","a":"=\t=","a1":"=\t="}]
> while CSVRecordSetWriter will produce:
>
> {code:java}
> case,a,a1
> tab,= =,= =
> {code}
> there is a actual "tab" in between "="
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Reopened] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura reopened NIFI-4621: - [~kislayom] Please mark a JIRA 'Resolved' when the change has been merged to the NiFi project. > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Peter Wicks >Priority: Critical > Fix For: 1.9.0 > > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-5874) CSVRecordSetWriter inject transformed backslash sequences from input
Ed Berezitsky created NIFI-5874:
---
Summary: CSVRecordSetWriter inject transformed backslash sequences
from input
Key: NIFI-5874
URL: https://issues.apache.org/jira/browse/NIFI-5874
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Affects Versions: 1.8.0
Reporter: Ed Berezitsky
If there is backslash sequence (like \t, \n, etc) in the input,
CSVRecordSetWriter transforms them into actual characters (new line, tab, etc)
in output record.
For example, input record:
```
case,a,a1
period,=\t=,-
```
Update Record with `/a1: /a` (just copy value from one field to another)
JsonRecordSetWriter will produce:
```
[\{"case":"period","a":"=\t=","a1":"=\t="}]
```
while CSVRecordSetWriter will produce:
```
case,a,a1
period,= =,= =
```
there is a actual "tab" in between "="
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (NIFI-5873) Listing FlowFiles only query the local partition of a SocketLoadBalancedFlowFileQueue
Truong Duc Kien created NIFI-5873:
-
Summary: Listing FlowFiles only query the local partition of a
SocketLoadBalancedFlowFileQueue
Key: NIFI-5873
URL: https://issues.apache.org/jira/browse/NIFI-5873
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 1.8.0
Reporter: Truong Duc Kien
Currently, a \{{SocketLoadBalancedFlowFileQueue}} only list the local partition
when a listing request is performed
[https://github.com/apache/nifi/blob/234ddb0fe1a36ad947c340114058d82c777d791f/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/queue/clustered/SocketLoadBalancedFlowFileQueue.java#L983]
This issue results in the inconsistency between the size of the queue (which
includes all partitions) and the list of FlowFiles return when performing a
listing operation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710839#comment-16710839 ] Kislay Kumar commented on NIFI-4621: Hi [~patricker] I also assumed (as the reported mentioned) initially that we might be missing EL on the processors properties. However I ended up finding that almost all properties are already EL enabled. It was only that the processor were not allowed to have input connection. If you would like i am happy to post some evidence for that. And i did test EL on the properties mentioned in the Jira. > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Peter Wicks >Priority: Critical > Fix For: 1.9.0 > > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5639) PublishAMQP Processor
[ https://issues.apache.org/jira/browse/NIFI-5639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710666#comment-16710666 ] Peter Wicks commented on NIFI-5639: --- [~kislayom] - Probably the most important thing is that you have a use for it or can easily test it. Personally, I have never used this processor before > PublishAMQP Processor > --- > > Key: NIFI-5639 > URL: https://issues.apache.org/jira/browse/NIFI-5639 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.5.0, 1.6.0, 1.7.0, 1.7.1 >Reporter: Mohammed Nadeem >Priority: Critical > Original Estimate: 96h > Remaining Estimate: 96h > > *PublishAMQP* Processor is routing incoming flowfile to success relationship > though invalid properties are configured. The processor is not throwing any > errors when invalid properties are configured at run-time. When the message > is not published with invalid properties, the processor silently routes the > message to success when it is suppose to route to failure relationship > indicating message was not published to the queue. This is a bug and no > proper error handling when failures occur *(nifi-amqp-bundle)* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4946) nifi-spark-bundle : Adding support for pyfiles, file, jars options
[
https://issues.apache.org/jira/browse/NIFI-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710643#comment-16710643
]
ASF GitHub Bot commented on NIFI-4946:
--
Github user mattyb149 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2521#discussion_r239241291
--- Diff:
nifi-nar-bundles/nifi-spark-bundle/nifi-livy-processors/src/main/java/org/apache/nifi/processors/livy/ExecuteSparkInteractive.java
---
@@ -178,48 +247,188 @@ public void onTrigger(ProcessContext context, final
ProcessSession session) thro
String sessionId = livyController.get("sessionId");
String livyUrl = livyController.get("livyUrl");
-String code =
context.getProperty(CODE).evaluateAttributeExpressions(flowFile).getValue();
-if (StringUtils.isEmpty(code)) {
-try (InputStream inputStream = session.read(flowFile)) {
-// If no code was provided, assume it is in the content of
the incoming flow file
-code = IOUtils.toString(inputStream, charset);
-} catch (IOException ioe) {
-log.error("Error reading input flowfile, penalizing and
routing to failure", new Object[]{flowFile, ioe.getMessage()}, ioe);
-flowFile = session.penalize(flowFile);
-session.transfer(flowFile, REL_FAILURE);
-return;
-}
-}
-code = StringEscapeUtils.escapeJavaScript(code);
-String payload = "{\"code\":\"" + code + "\"}";
+
try {
-final JSONObject result = submitAndHandleJob(livyUrl,
livySessionService, sessionId, payload, statusCheckInterval);
-log.debug("ExecuteSparkInteractive Result of Job Submit: " +
result);
-if (result == null) {
-session.transfer(flowFile, REL_FAILURE);
-} else {
+
+if (isBatchJob) {
+
+String jsonResponse = null;
+
+if (StringUtils.isEmpty(jsonResponse)) {
+try (InputStream inputStream = session.read(flowFile))
{
+// If no code was provided, assume it is in the
content of the incoming flow file
+jsonResponse = IOUtils.toString(inputStream,
charset);
+} catch (IOException ioe) {
+log.error("Error reading input flowfile,
penalizing and routing to failure", new Object[]{flowFile, ioe.getMessage()},
ioe);
+flowFile = session.penalize(flowFile);
+session.transfer(flowFile, REL_FAILURE);
+return;
+}
+}
+
+log.debug(" > jsonResponse: " + jsonResponse);
+
try {
-final JSONObject output = result.getJSONObject("data");
-flowFile = session.write(flowFile, out ->
out.write(output.toString().getBytes()));
-flowFile = session.putAttribute(flowFile,
CoreAttributes.MIME_TYPE.key(), LivySessionService.APPLICATION_JSON);
-session.transfer(flowFile, REL_SUCCESS);
-} catch (JSONException je) {
-// The result doesn't contain the data, just send the
output object as the flow file content to failure (after penalizing)
-log.error("Spark Session returned an error, sending
the output JSON object as the flow file content to failure (after penalizing)");
-flowFile = session.write(flowFile, out ->
out.write(result.toString().getBytes()));
+
+final JSONObject jsonResponseObj = new
JSONObject(jsonResponse);
+
+Map headers = new HashMap<>();
+headers.put("Content-Type",
LivySessionService.APPLICATION_JSON);
+headers.put("X-Requested-By", LivySessionService.USER);
+headers.put("Accept", "application/json");
+
+JSONObject jobInfo =
readJSONObjectFromUrl(jsonResponseObj.getString("url"), livySessionService,
headers);
+
+flowFile = session.write(flowFile, out ->
out.write(jobInfo.toString().getBytes()));
flowFile = session.putAttribute(flowFile,
CoreAttributes.MIME_TYPE.key(), LivySessionService.APPLICATION_JSON);
-flowFile = session.penalize(flowFile);
+
+Thread.sleep(statusCheckInterval);
+
+String state = jobInfo.getString("state");
+log.debug(" > jsonResponseObj State: " + state);
+
+
[GitHub] nifi pull request #2521: NIFI-4946 nifi-spark-bundle : Adding support for py...
Github user mattyb149 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2521#discussion_r239241291
--- Diff:
nifi-nar-bundles/nifi-spark-bundle/nifi-livy-processors/src/main/java/org/apache/nifi/processors/livy/ExecuteSparkInteractive.java
---
@@ -178,48 +247,188 @@ public void onTrigger(ProcessContext context, final
ProcessSession session) thro
String sessionId = livyController.get("sessionId");
String livyUrl = livyController.get("livyUrl");
-String code =
context.getProperty(CODE).evaluateAttributeExpressions(flowFile).getValue();
-if (StringUtils.isEmpty(code)) {
-try (InputStream inputStream = session.read(flowFile)) {
-// If no code was provided, assume it is in the content of
the incoming flow file
-code = IOUtils.toString(inputStream, charset);
-} catch (IOException ioe) {
-log.error("Error reading input flowfile, penalizing and
routing to failure", new Object[]{flowFile, ioe.getMessage()}, ioe);
-flowFile = session.penalize(flowFile);
-session.transfer(flowFile, REL_FAILURE);
-return;
-}
-}
-code = StringEscapeUtils.escapeJavaScript(code);
-String payload = "{\"code\":\"" + code + "\"}";
+
try {
-final JSONObject result = submitAndHandleJob(livyUrl,
livySessionService, sessionId, payload, statusCheckInterval);
-log.debug("ExecuteSparkInteractive Result of Job Submit: " +
result);
-if (result == null) {
-session.transfer(flowFile, REL_FAILURE);
-} else {
+
+if (isBatchJob) {
+
+String jsonResponse = null;
+
+if (StringUtils.isEmpty(jsonResponse)) {
+try (InputStream inputStream = session.read(flowFile))
{
+// If no code was provided, assume it is in the
content of the incoming flow file
+jsonResponse = IOUtils.toString(inputStream,
charset);
+} catch (IOException ioe) {
+log.error("Error reading input flowfile,
penalizing and routing to failure", new Object[]{flowFile, ioe.getMessage()},
ioe);
+flowFile = session.penalize(flowFile);
+session.transfer(flowFile, REL_FAILURE);
+return;
+}
+}
+
+log.debug(" > jsonResponse: " + jsonResponse);
+
try {
-final JSONObject output = result.getJSONObject("data");
-flowFile = session.write(flowFile, out ->
out.write(output.toString().getBytes()));
-flowFile = session.putAttribute(flowFile,
CoreAttributes.MIME_TYPE.key(), LivySessionService.APPLICATION_JSON);
-session.transfer(flowFile, REL_SUCCESS);
-} catch (JSONException je) {
-// The result doesn't contain the data, just send the
output object as the flow file content to failure (after penalizing)
-log.error("Spark Session returned an error, sending
the output JSON object as the flow file content to failure (after penalizing)");
-flowFile = session.write(flowFile, out ->
out.write(result.toString().getBytes()));
+
+final JSONObject jsonResponseObj = new
JSONObject(jsonResponse);
+
+Map headers = new HashMap<>();
+headers.put("Content-Type",
LivySessionService.APPLICATION_JSON);
+headers.put("X-Requested-By", LivySessionService.USER);
+headers.put("Accept", "application/json");
+
+JSONObject jobInfo =
readJSONObjectFromUrl(jsonResponseObj.getString("url"), livySessionService,
headers);
+
+flowFile = session.write(flowFile, out ->
out.write(jobInfo.toString().getBytes()));
flowFile = session.putAttribute(flowFile,
CoreAttributes.MIME_TYPE.key(), LivySessionService.APPLICATION_JSON);
-flowFile = session.penalize(flowFile);
+
+Thread.sleep(statusCheckInterval);
+
+String state = jobInfo.getString("state");
+log.debug(" > jsonResponseObj State: " + state);
+
+switch (state) {
+case "success":
+log.debug(" > success State: " + state);
+session.transfer(flowFile, REL_SUCCESS);
+
[jira] [Commented] (NIFI-1364) Audit OCSP certificate validation
[ https://issues.apache.org/jira/browse/NIFI-1364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710583#comment-16710583 ] ASF GitHub Bot commented on NIFI-1364: -- GitHub user thenatog opened a pull request: https://github.com/apache/nifi/pull/3204 NIFI-1364 - Removed custom OCSP certificate revocation checking code and replaced with just using Java native implementation. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/thenatog/nifi ocsp_changes Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3204.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3204 commit ae351199c0b0a1edd54670846e1aef45427ca1a7 Author: thenatog Date: 2018-11-16T02:52:05Z OCSP revocation checking working commit 143ab823bdcd58830a11ab17e655e5ee7ba34e09 Author: thenatog Date: 2018-11-21T15:32:05Z Added OCSP enable setting to NiFi properties. commit 4dd438fe02f5968e70382d43c9c01c1f003b6a94 Author: thenatog Date: 2018-12-03T14:44:53Z Added groovy tests commit 0d3f837a4aa11517e0fca875914a99cbed20dde2 Author: thenatog Date: 2018-12-03T19:13:16Z Fixed some tests commit 35e45e080e353c55d47e22400df285fb7cc65f0d Author: thenatog Date: 2018-12-03T20:17:55Z Removed some references to old OCSP cert validator commit a85a10656e95369e97fd88ad3c0c8a29481b654f Author: thenatog Date: 2018-12-03T22:03:55Z Backed out changes to the SslContextFactory used for SslContextService commit 932d32480ca82e00ca71314a2fdc3f74ac3eeb14 Author: thenatog Date: 2018-12-04T15:25:20Z Minor changes commit 0b50a806d29ea274c24e9be4c7a8f97641fedd3a Author: thenatog Date: 2018-12-05T03:08:30Z Small changes commit 0d62efacf3c3f8cf551b2020fe3dd233e3e920d9 Author: thenatog Date: 2018-12-05T15:52:47Z Set custom responder URL if it's set in NiFiProperties. > Audit OCSP certificate validation > - > > Key: NIFI-1364 > URL: https://issues.apache.org/jira/browse/NIFI-1364 > Project: Apache NiFi > Issue Type: Task > Components: Core Framework >Affects Versions: 0.4.1 >Reporter: Andy LoPresto >Assignee: Nathan Gough >Priority: Major > Labels: certificate, ocsp, security > > While upgrading the version of BouncyCastle libraries used, I had to re-write > the OCSP certificate validation code because BC split the PKIX code into a > separate module and renamed many classes & methods. During this re-write, I > made the code compile using the new logic, but I am unsure that OCSP > validation needs to occur outside of the SSL/TLS negotiation, or that the > current mechanism is correct. > Questions: > * Can we use Java's built-in OCSP validation? [1][2] > * Is the current mechanism correct, where a local cache is used with custom > internal classes representing OCSP requests and statuses, and it queries a
[jira] [Updated] (NIFI-5718) Performance degraded in ReplaceText processor
[ https://issues.apache.org/jira/browse/NIFI-5718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-5718: - Resolution: Fixed Fix Version/s: 1.9.0 Status: Resolved (was: Patch Available) > Performance degraded in ReplaceText processor > - > > Key: NIFI-5718 > URL: https://issues.apache.org/jira/browse/NIFI-5718 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.8.0 >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.9.0 > > Attachments: Screen Shot 2018-10-17 at 10.55.53 AM.png > > > NIFI-5711 addresses some licensing concerns in the NLKBufferedReader class. > In doing so, however, it results in lower performance. The ReplaceText > processor is affected if the Evaluation Mode is set to Line-by-Line, and the > RouteText processor will also be affected. We should be able to match the > performance of the previous version. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #3204: NIFI-1364 - Removed custom OCSP certificate revocat...
GitHub user thenatog opened a pull request: https://github.com/apache/nifi/pull/3204 NIFI-1364 - Removed custom OCSP certificate revocation checking code and replaced with just using Java native implementation. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/thenatog/nifi ocsp_changes Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3204.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3204 commit ae351199c0b0a1edd54670846e1aef45427ca1a7 Author: thenatog Date: 2018-11-16T02:52:05Z OCSP revocation checking working commit 143ab823bdcd58830a11ab17e655e5ee7ba34e09 Author: thenatog Date: 2018-11-21T15:32:05Z Added OCSP enable setting to NiFi properties. commit 4dd438fe02f5968e70382d43c9c01c1f003b6a94 Author: thenatog Date: 2018-12-03T14:44:53Z Added groovy tests commit 0d3f837a4aa11517e0fca875914a99cbed20dde2 Author: thenatog Date: 2018-12-03T19:13:16Z Fixed some tests commit 35e45e080e353c55d47e22400df285fb7cc65f0d Author: thenatog Date: 2018-12-03T20:17:55Z Removed some references to old OCSP cert validator commit a85a10656e95369e97fd88ad3c0c8a29481b654f Author: thenatog Date: 2018-12-03T22:03:55Z Backed out changes to the SslContextFactory used for SslContextService commit 932d32480ca82e00ca71314a2fdc3f74ac3eeb14 Author: thenatog Date: 2018-12-04T15:25:20Z Minor changes commit 0b50a806d29ea274c24e9be4c7a8f97641fedd3a Author: thenatog Date: 2018-12-05T03:08:30Z Small changes commit 0d62efacf3c3f8cf551b2020fe3dd233e3e920d9 Author: thenatog Date: 2018-12-05T15:52:47Z Set custom responder URL if it's set in NiFiProperties. ---
[jira] [Commented] (NIFI-5859) Update NAR maven plugin to include information about Extensions
[ https://issues.apache.org/jira/browse/NIFI-5859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710582#comment-16710582 ] ASF GitHub Bot commented on NIFI-5859: -- Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/3192 OK @bbende I have pushed new commits that I believe address all of the concerns above. > Update NAR maven plugin to include information about Extensions > --- > > Key: NIFI-5859 > URL: https://issues.apache.org/jira/browse/NIFI-5859 > Project: Apache NiFi > Issue Type: New Feature > Components: Tools and Build >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > > In order to have the NiFi Registry host any extensions, the registry will > need a way to know what extensions exist in a given NAR. Currently, that > information is not available directly. > The NAR maven plugin should be updated to provide a list of extensions and > for each one, provide at least the following minimal information: > * Extension Type > * Extension Name > * Capability Description > * Whether or not the component is Restricted, "sub-restrictions" it has, and > explanations of both > * Any Tags that the component has > * If the component is a Controller Service, any Controller Service API's > that it provides > Additionally, it would be ideal to provide all documentation for the > component within the NAR. It is best, though, not to write the documentation > in HTML as is done now but rather in XML or some sort of form that provides > the information in a structured way without any styling. This would allow the > documentation to be rendered consistently, even if the styling changes from 1 > version to the next. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #3192: NIFI-5859: Added XML-based documentation writer that can b...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/3192 OK @bbende I have pushed new commits that I believe address all of the concerns above. ---
[jira] [Commented] (MINIFICPP-689) Make minifi::Exception constructible with string param
[ https://issues.apache.org/jira/browse/MINIFICPP-689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710560#comment-16710560 ] ASF GitHub Bot commented on MINIFICPP-689: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/455 > Make minifi::Exception constructible with string param > -- > > Key: MINIFICPP-689 > URL: https://issues.apache.org/jira/browse/MINIFICPP-689 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Minor > Fix For: 0.6.0 > > > Exception is currently only constructible using const char * argument, but > that's copied into a string. Using string parameter would make it more > developer-friendly and save some copy constructions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #455: MINIFICPP-689 - Make minifi::Exception co...
Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/455 ---
[GitHub] nifi-minifi pull request #148: MINIFI-482 Provide support for multiple URIs.
Github user jzonthemtn commented on a diff in the pull request:
https://github.com/apache/nifi-minifi/pull/148#discussion_r239213358
--- Diff:
minifi-commons/minifi-commons-schema/src/main/java/org/apache/nifi/minifi/commons/schema/RemoteProcessGroupSchema.java
---
@@ -151,7 +152,7 @@ public String getComment() {
return comment;
}
-public String getUrl() {
+public String getUrls() {
return url;
--- End diff --
Super minor and I honestly didn't look too close to see if it makes sense,
but would it be beneficial at all to rename `url` to `urls`, too?
---
[jira] [Commented] (MINIFICPP-404) HTTP Proxy Support for HTTP Site to Site
[ https://issues.apache.org/jira/browse/MINIFICPP-404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710555#comment-16710555 ] ASF GitHub Bot commented on MINIFICPP-404: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/454 > HTTP Proxy Support for HTTP Site to Site > > > Key: MINIFICPP-404 > URL: https://issues.apache.org/jira/browse/MINIFICPP-404 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.6.0 >Reporter: bqiu >Assignee: Mr TheSegfault >Priority: Minor > Fix For: 0.6.0 > > > HTTP Proxy Support for HTTP Site to Site > http://nifi.apache.org/docs/nifi-docs/html/user-guide.html#configure-site-to-site-client-nifi-instance. > support for this in YAML config via > https://github.com/apache/nifi-minifi/blob/master/minifi-docs/src/main/markdown/System_Admin_Guide.md#remote-process-groups-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #454: MINIFICPP-404: Correct invalid assumption...
Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/454 ---
[jira] [Assigned] (NIFI-5872) Add compression option on JsonRecordSetWriter
[ https://issues.apache.org/jira/browse/NIFI-5872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard reassigned NIFI-5872: Assignee: Pierre Villard > Add compression option on JsonRecordSetWriter > - > > Key: NIFI-5872 > URL: https://issues.apache.org/jira/browse/NIFI-5872 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > > Add a compression option on the JsonRecordSetWriter controller service as we > have on the AvroRecordSetWriter controller service. This can be useful in > case of very IO intensive workflows. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5826) UpdateRecord processor throwing PatternSyntaxException
[
https://issues.apache.org/jira/browse/NIFI-5826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710525#comment-16710525
]
ASF GitHub Bot commented on NIFI-5826:
--
Github user ottobackwards commented on the issue:
https://github.com/apache/nifi/pull/3200
There should be new tests that go along with this
> UpdateRecord processor throwing PatternSyntaxException
> --
>
> Key: NIFI-5826
> URL: https://issues.apache.org/jira/browse/NIFI-5826
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0, 1.7.0, 1.8.0, 1.7.1
> Environment: Nifi in docker container
>Reporter: ravi kargam
>Assignee: Ed Berezitsky
>Priority: Minor
> Attachments: NIFI-5826_PR-3183.patch,
> UpdateRecord_Config_Exception.JPG
>
>
> with replacement value strategy as Record Path Value,
> I am trying to replace square bracket symbol *[* with parenthesis symbol *(*
> in my employeeName column in my csvReader structure with below syntax
> replaceRegex(/employeeName, "[\[]", "(")
> Processor is throwing following exception.
> RecordPathException: java.util.regex.PatternSyntaxException: Unclosed
> character class near index 4 [\\[]
> It worked fine with other special characters such as \{, }, <, >, ;, _, "
> For double qoute ("), i had to use single escape character, for above listed
> other characters, worked fine without any escape character. Other folks in
> Nifi Slack tried \s, \d, \w, \.
> looks like non of them worked.
> replace function worked for replacing [ and ]characters. didn't test any
> other characters.
> Please address resolve the issue.
> Regards,
> Ravi
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi issue #3200: NIFI-5826 WIP Fix back-slash escaping at Lexers
Github user ottobackwards commented on the issue: https://github.com/apache/nifi/pull/3200 There should be new tests that go along with this ---
[jira] [Commented] (NIFI-5871) MockProcessSession.putAllAttributes should ignore the UUID attribute
[ https://issues.apache.org/jira/browse/NIFI-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710440#comment-16710440 ] ASF GitHub Bot commented on NIFI-5871: -- Github user SavtechSolutions commented on the issue: https://github.com/apache/nifi/pull/3203 @patricker Yes, I'm inclined to agree - the enqueue() call with attributes uses the same putAllAttributes call under the hood, so there's no way it should be able to retain the UUID attribute. I'll get to fix the tests, thanks. > MockProcessSession.putAllAttributes should ignore the UUID attribute > > > Key: NIFI-5871 > URL: https://issues.apache.org/jira/browse/NIFI-5871 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.8.0 >Reporter: Alex Savitsky >Priority: Major > > Currently, the method would copy all attributes indiscriminately, but the > interface Javadoc specifically states that the attribute "uuid" should be > ignored. This leads to issues with testing, where two distinct flow files are > considered same in the session. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #3203: NIFI-5871 ignore UUID attribute when copying flow file att...
Github user patricker commented on the issue: https://github.com/apache/nifi/pull/3203 @MikeThomsen I think the test is showing that this isn't a parent/child scenario, but showing that it's the same FlowFile from start to finish. If there were a way to inject content into `MockFlowFile`, then we could build one, pass that in to `enqueue` and the test would actually work. ---
[jira] [Commented] (NIFI-5871) MockProcessSession.putAllAttributes should ignore the UUID attribute
[ https://issues.apache.org/jira/browse/NIFI-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710449#comment-16710449 ] ASF GitHub Bot commented on NIFI-5871: -- Github user patricker commented on the issue: https://github.com/apache/nifi/pull/3203 @MikeThomsen I think the test is showing that this isn't a parent/child scenario, but showing that it's the same FlowFile from start to finish. If there were a way to inject content into `MockFlowFile`, then we could build one, pass that in to `enqueue` and the test would actually work. > MockProcessSession.putAllAttributes should ignore the UUID attribute > > > Key: NIFI-5871 > URL: https://issues.apache.org/jira/browse/NIFI-5871 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.8.0 >Reporter: Alex Savitsky >Priority: Major > > Currently, the method would copy all attributes indiscriminately, but the > interface Javadoc specifically states that the attribute "uuid" should be > ignored. This leads to issues with testing, where two distinct flow files are > considered same in the session. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5871) MockProcessSession.putAllAttributes should ignore the UUID attribute
[ https://issues.apache.org/jira/browse/NIFI-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710446#comment-16710446 ] ASF GitHub Bot commented on NIFI-5871: -- Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3203 AFAIK, there is no reason why uuid should be inherited by a flowfile even in a parent-child scenario since the child is a new flowfile branched off. > MockProcessSession.putAllAttributes should ignore the UUID attribute > > > Key: NIFI-5871 > URL: https://issues.apache.org/jira/browse/NIFI-5871 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.8.0 >Reporter: Alex Savitsky >Priority: Major > > Currently, the method would copy all attributes indiscriminately, but the > interface Javadoc specifically states that the attribute "uuid" should be > ignored. This leads to issues with testing, where two distinct flow files are > considered same in the session. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #3203: NIFI-5871 ignore UUID attribute when copying flow file att...
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3203 AFAIK, there is no reason why uuid should be inherited by a flowfile even in a parent-child scenario since the child is a new flowfile branched off. ---
[jira] [Commented] (NIFI-5871) MockProcessSession.putAllAttributes should ignore the UUID attribute
[ https://issues.apache.org/jira/browse/NIFI-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710408#comment-16710408 ] ASF GitHub Bot commented on NIFI-5871: -- Github user SavtechSolutions commented on the issue: https://github.com/apache/nifi/pull/3203 Now this is strange - is there a remote chance that this behavior (copying UUID) is expected and it's not a bug? Or is this just an unit test with incorrect assumptions? > MockProcessSession.putAllAttributes should ignore the UUID attribute > > > Key: NIFI-5871 > URL: https://issues.apache.org/jira/browse/NIFI-5871 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.8.0 >Reporter: Alex Savitsky >Priority: Major > > Currently, the method would copy all attributes indiscriminately, but the > interface Javadoc specifically states that the attribute "uuid" should be > ignored. This leads to issues with testing, where two distinct flow files are > considered same in the session. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #3203: NIFI-5871 ignore UUID attribute when copying flow file att...
Github user SavtechSolutions commented on the issue: https://github.com/apache/nifi/pull/3203 @patricker Yes, I'm inclined to agree - the enqueue() call with attributes uses the same putAllAttributes call under the hood, so there's no way it should be able to retain the UUID attribute. I'll get to fix the tests, thanks. ---
[jira] [Commented] (NIFI-5868) Instrument robust timing information for ListFile
[ https://issues.apache.org/jira/browse/NIFI-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710424#comment-16710424 ] ASF GitHub Bot commented on NIFI-5868: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/3202 > Instrument robust timing information for ListFile > - > > Key: NIFI-5868 > URL: https://issues.apache.org/jira/browse/NIFI-5868 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.9.0 > > > ListFile is used in many different contexts. We often see users with a > specific use case, though, which is to run ListFile on a Primary Node in a > cluster, in order to obtain a file listing of an NFS-mounted share. This > works well in most cases, but whenever problems do arise, it is very > difficult to understand what the problem is. It would be very helpful to have > information such as: > * Is there a problem accessing a specific file on the NFS mount? > * Is there a problem obtaining a listing from the NFS mount? > * Is progress being made at all? > * How long is a listing taking right now? > * How long does a listing typically take? > * Is this problem related to NiFi or to the operating system / > infrastructure? > It would be helpful to track information about each disk access that is > occurring, as well as the overall listing progress and issue warnings if we > see clear problems arise. We can do this by timing how long each disk access > takes, what file was being accessed, and what operating was being performed. > If we capture this data in a rolling window, we can assess the data to > determine if the listing is now taking longer than it did previously and > alert to this fact. Or alert if performing a specific disk operation is > taking a long time. > Gathering this information will likely be fairly heap intensive, so it is > best to make the functionality optional. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5871) MockProcessSession.putAllAttributes should ignore the UUID attribute
[ https://issues.apache.org/jira/browse/NIFI-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710427#comment-16710427 ] ASF GitHub Bot commented on NIFI-5871: -- Github user patricker commented on the issue: https://github.com/apache/nifi/pull/3203 @SavtechSolutions Yes, this is a bug in the tests I think. I can see they manually provide a custom `UUID` instead of letting NiFi generate it. > MockProcessSession.putAllAttributes should ignore the UUID attribute > > > Key: NIFI-5871 > URL: https://issues.apache.org/jira/browse/NIFI-5871 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.8.0 >Reporter: Alex Savitsky >Priority: Major > > Currently, the method would copy all attributes indiscriminately, but the > interface Javadoc specifically states that the attribute "uuid" should be > ignored. This leads to issues with testing, where two distinct flow files are > considered same in the session. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5826) UpdateRecord processor throwing PatternSyntaxException
[
https://issues.apache.org/jira/browse/NIFI-5826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710412#comment-16710412
]
ASF GitHub Bot commented on NIFI-5826:
--
Github user bdesert commented on the issue:
https://github.com/apache/nifi/pull/3200
Reviewing...
> UpdateRecord processor throwing PatternSyntaxException
> --
>
> Key: NIFI-5826
> URL: https://issues.apache.org/jira/browse/NIFI-5826
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0, 1.7.0, 1.8.0, 1.7.1
> Environment: Nifi in docker container
>Reporter: ravi kargam
>Assignee: Ed Berezitsky
>Priority: Minor
> Attachments: NIFI-5826_PR-3183.patch,
> UpdateRecord_Config_Exception.JPG
>
>
> with replacement value strategy as Record Path Value,
> I am trying to replace square bracket symbol *[* with parenthesis symbol *(*
> in my employeeName column in my csvReader structure with below syntax
> replaceRegex(/employeeName, "[\[]", "(")
> Processor is throwing following exception.
> RecordPathException: java.util.regex.PatternSyntaxException: Unclosed
> character class near index 4 [\\[]
> It worked fine with other special characters such as \{, }, <, >, ;, _, "
> For double qoute ("), i had to use single escape character, for above listed
> other characters, worked fine without any escape character. Other folks in
> Nifi Slack tried \s, \d, \w, \.
> looks like non of them worked.
> replace function worked for replacing [ and ]characters. didn't test any
> other characters.
> Please address resolve the issue.
> Regards,
> Ravi
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-5868) Instrument robust timing information for ListFile
[ https://issues.apache.org/jira/browse/NIFI-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende updated NIFI-5868: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Instrument robust timing information for ListFile > - > > Key: NIFI-5868 > URL: https://issues.apache.org/jira/browse/NIFI-5868 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.9.0 > > > ListFile is used in many different contexts. We often see users with a > specific use case, though, which is to run ListFile on a Primary Node in a > cluster, in order to obtain a file listing of an NFS-mounted share. This > works well in most cases, but whenever problems do arise, it is very > difficult to understand what the problem is. It would be very helpful to have > information such as: > * Is there a problem accessing a specific file on the NFS mount? > * Is there a problem obtaining a listing from the NFS mount? > * Is progress being made at all? > * How long is a listing taking right now? > * How long does a listing typically take? > * Is this problem related to NiFi or to the operating system / > infrastructure? > It would be helpful to track information about each disk access that is > occurring, as well as the overall listing progress and issue warnings if we > see clear problems arise. We can do this by timing how long each disk access > takes, what file was being accessed, and what operating was being performed. > If we capture this data in a rolling window, we can assess the data to > determine if the listing is now taking longer than it did previously and > alert to this fact. Or alert if performing a specific disk operation is > taking a long time. > Gathering this information will likely be fairly heap intensive, so it is > best to make the functionality optional. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #3203: NIFI-5871 ignore UUID attribute when copying flow file att...
Github user patricker commented on the issue: https://github.com/apache/nifi/pull/3203 @SavtechSolutions Yes, this is a bug in the tests I think. I can see they manually provide a custom `UUID` instead of letting NiFi generate it. ---
[jira] [Commented] (NIFI-5868) Instrument robust timing information for ListFile
[ https://issues.apache.org/jira/browse/NIFI-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710423#comment-16710423 ] ASF subversion and git services commented on NIFI-5868: --- Commit 72ea93a657cd673bfc40ef74e172f5efb950e95d in nifi's branch refs/heads/master from [~markap14] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=72ea93a ] NIFI-5868: Added instrumentation around ListFile such that all disk accesses are timed and any unusually long listing times or disk access operations can be logged. Additionally, information is logged at a debug level including significant amounts of troubleshooting information when configured to do so This closes #3202. Signed-off-by: Bryan Bende > Instrument robust timing information for ListFile > - > > Key: NIFI-5868 > URL: https://issues.apache.org/jira/browse/NIFI-5868 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.9.0 > > > ListFile is used in many different contexts. We often see users with a > specific use case, though, which is to run ListFile on a Primary Node in a > cluster, in order to obtain a file listing of an NFS-mounted share. This > works well in most cases, but whenever problems do arise, it is very > difficult to understand what the problem is. It would be very helpful to have > information such as: > * Is there a problem accessing a specific file on the NFS mount? > * Is there a problem obtaining a listing from the NFS mount? > * Is progress being made at all? > * How long is a listing taking right now? > * How long does a listing typically take? > * Is this problem related to NiFi or to the operating system / > infrastructure? > It would be helpful to track information about each disk access that is > occurring, as well as the overall listing progress and issue warnings if we > see clear problems arise. We can do this by timing how long each disk access > takes, what file was being accessed, and what operating was being performed. > If we capture this data in a rolling window, we can assess the data to > determine if the listing is now taking longer than it did previously and > alert to this fact. Or alert if performing a specific disk operation is > taking a long time. > Gathering this information will likely be fairly heap intensive, so it is > best to make the functionality optional. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #3202: NIFI-5868: Added instrumentation around ListFile su...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/3202 ---
[GitHub] nifi issue #3202: NIFI-5868: Added instrumentation around ListFile such that...
Github user bbende commented on the issue: https://github.com/apache/nifi/pull/3202 Code looks good, verified I can see the debug logging when enabled, going to merge ---
[jira] [Commented] (NIFI-5868) Instrument robust timing information for ListFile
[ https://issues.apache.org/jira/browse/NIFI-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710417#comment-16710417 ] ASF GitHub Bot commented on NIFI-5868: -- Github user bbende commented on the issue: https://github.com/apache/nifi/pull/3202 Code looks good, verified I can see the debug logging when enabled, going to merge > Instrument robust timing information for ListFile > - > > Key: NIFI-5868 > URL: https://issues.apache.org/jira/browse/NIFI-5868 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.9.0 > > > ListFile is used in many different contexts. We often see users with a > specific use case, though, which is to run ListFile on a Primary Node in a > cluster, in order to obtain a file listing of an NFS-mounted share. This > works well in most cases, but whenever problems do arise, it is very > difficult to understand what the problem is. It would be very helpful to have > information such as: > * Is there a problem accessing a specific file on the NFS mount? > * Is there a problem obtaining a listing from the NFS mount? > * Is progress being made at all? > * How long is a listing taking right now? > * How long does a listing typically take? > * Is this problem related to NiFi or to the operating system / > infrastructure? > It would be helpful to track information about each disk access that is > occurring, as well as the overall listing progress and issue warnings if we > see clear problems arise. We can do this by timing how long each disk access > takes, what file was being accessed, and what operating was being performed. > If we capture this data in a rolling window, we can assess the data to > determine if the listing is now taking longer than it did previously and > alert to this fact. Or alert if performing a specific disk operation is > taking a long time. > Gathering this information will likely be fairly heap intensive, so it is > best to make the functionality optional. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #3200: NIFI-5826 WIP Fix back-slash escaping at Lexers
Github user bdesert commented on the issue: https://github.com/apache/nifi/pull/3200 Reviewing... ---
[GitHub] nifi issue #3203: NIFI-5871 ignore UUID attribute when copying flow file att...
Github user SavtechSolutions commented on the issue: https://github.com/apache/nifi/pull/3203 Now this is strange - is there a remote chance that this behavior (copying UUID) is expected and it's not a bug? Or is this just an unit test with incorrect assumptions? ---
[GitHub] nifi-minifi pull request #147: MINIFI-483: Adding Ubuntu 16/18 as supported ...
Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi/pull/147 ---
[jira] [Commented] (NIFI-5871) MockProcessSession.putAllAttributes should ignore the UUID attribute
[ https://issues.apache.org/jira/browse/NIFI-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710395#comment-16710395 ] ASF GitHub Bot commented on NIFI-5871: -- Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3203 Approval still stands. Will wait to see what Travis does. > MockProcessSession.putAllAttributes should ignore the UUID attribute > > > Key: NIFI-5871 > URL: https://issues.apache.org/jira/browse/NIFI-5871 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.8.0 >Reporter: Alex Savitsky >Priority: Major > > Currently, the method would copy all attributes indiscriminately, but the > interface Javadoc specifically states that the attribute "uuid" should be > ignored. This leads to issues with testing, where two distinct flow files are > considered same in the session. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #3203: NIFI-5871 ignore UUID attribute when copying flow file att...
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3203 Approval still stands. Will wait to see what Travis does. ---
[jira] [Commented] (NIFI-5871) MockProcessSession.putAllAttributes should ignore the UUID attribute
[ https://issues.apache.org/jira/browse/NIFI-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710393#comment-16710393 ] ASF GitHub Bot commented on NIFI-5871: -- Github user SavtechSolutions commented on the issue: https://github.com/apache/nifi/pull/3203 @patricker I've incorporated your feedback > MockProcessSession.putAllAttributes should ignore the UUID attribute > > > Key: NIFI-5871 > URL: https://issues.apache.org/jira/browse/NIFI-5871 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.8.0 >Reporter: Alex Savitsky >Priority: Major > > Currently, the method would copy all attributes indiscriminately, but the > interface Javadoc specifically states that the attribute "uuid" should be > ignored. This leads to issues with testing, where two distinct flow files are > considered same in the session. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #3203: NIFI-5871 ignore UUID attribute when copying flow file att...
Github user SavtechSolutions commented on the issue: https://github.com/apache/nifi/pull/3203 @patricker I've incorporated your feedback ---
[jira] [Commented] (NIFI-5871) MockProcessSession.putAllAttributes should ignore the UUID attribute
[ https://issues.apache.org/jira/browse/NIFI-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710365#comment-16710365 ] ASF GitHub Bot commented on NIFI-5871: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3203#discussion_r239160879 --- Diff: nifi-mock/src/main/java/org/apache/nifi/util/MockProcessSession.java --- @@ -496,7 +496,10 @@ public MockFlowFile putAllAttributes(FlowFile flowFile, final Map attrCopy = new HashMap<>(); +attrCopy.putAll(attrs); --- End diff -- Only change I'd make here is putting that into the constructor. Other than that LGTM. > MockProcessSession.putAllAttributes should ignore the UUID attribute > > > Key: NIFI-5871 > URL: https://issues.apache.org/jira/browse/NIFI-5871 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.8.0 >Reporter: Alex Savitsky >Priority: Major > > Currently, the method would copy all attributes indiscriminately, but the > interface Javadoc specifically states that the attribute "uuid" should be > ignored. This leads to issues with testing, where two distinct flow files are > considered same in the session. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5621) Create a Connection Pooling service implementation to be used in Cassandra processors
[ https://issues.apache.org/jira/browse/NIFI-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710382#comment-16710382 ] ASF GitHub Bot commented on NIFI-5621: -- Github user zenfenan commented on the issue: https://github.com/apache/nifi/pull/3105 @MikeThomsen I have worked on the LICENSE & NOTICE files as suggested. Please take a look now. > Create a Connection Pooling service implementation to be used in Cassandra > processors > - > > Key: NIFI-5621 > URL: https://issues.apache.org/jira/browse/NIFI-5621 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Sivaprasanna Sethuraman >Assignee: Sivaprasanna Sethuraman >Priority: Major > > Like how the Relational Database processors leverage 'DBCPConnectionPool' > controller service, there should be one that could be used by the processors > from Cassandra bundle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #3105: NIFI-5621: Added Cassandra connection provider service
Github user zenfenan commented on the issue: https://github.com/apache/nifi/pull/3105 @MikeThomsen I have worked on the LICENSE & NOTICE files as suggested. Please take a look now. ---
[jira] [Commented] (NIFI-5621) Create a Connection Pooling service implementation to be used in Cassandra processors
[ https://issues.apache.org/jira/browse/NIFI-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710380#comment-16710380 ] ASF GitHub Bot commented on NIFI-5621: -- Github user zenfenan commented on a diff in the pull request: https://github.com/apache/nifi/pull/3105#discussion_r239163988 --- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-services-nar/src/main/resources/META-INF/NOTICE --- @@ -0,0 +1,327 @@ +nifi-cassandra-services-nar --- End diff -- Fixed all. Please take a look. > Create a Connection Pooling service implementation to be used in Cassandra > processors > - > > Key: NIFI-5621 > URL: https://issues.apache.org/jira/browse/NIFI-5621 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Sivaprasanna Sethuraman >Assignee: Sivaprasanna Sethuraman >Priority: Major > > Like how the Relational Database processors leverage 'DBCPConnectionPool' > controller service, there should be one that could be used by the processors > from Cassandra bundle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #3105: NIFI-5621: Added Cassandra connection provider serv...
Github user zenfenan commented on a diff in the pull request: https://github.com/apache/nifi/pull/3105#discussion_r239164057 --- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-services-nar/src/main/resources/META-INF/NOTICE --- @@ -0,0 +1,327 @@ +nifi-cassandra-services-nar +Copyright 2016 The Apache Software Foundation + +This product includes software developed at +The Apache Software Foundation (http://www.apache.org/). + +** +Apache Software License v2 +** + +The following binary components are provided under the Apache Software License v2 + + (ASLv2) DataStax Java Driver for Apache Cassandra - Core + The following NOTICE information applies: +DataStax Java Driver for Apache Cassandra - Core +Copyright (C) 2012-2017 DataStax Inc. + + (ASLv2) Apache Avro + The following NOTICE information applies: +Apache Avro +Copyright 2009-2017 The Apache Software Foundation + + (ASLv2) Jackson JSON processor --- End diff -- Understood. ---
[jira] [Commented] (NIFI-5621) Create a Connection Pooling service implementation to be used in Cassandra processors
[ https://issues.apache.org/jira/browse/NIFI-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710381#comment-16710381 ] ASF GitHub Bot commented on NIFI-5621: -- Github user zenfenan commented on a diff in the pull request: https://github.com/apache/nifi/pull/3105#discussion_r239164057 --- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-services-nar/src/main/resources/META-INF/NOTICE --- @@ -0,0 +1,327 @@ +nifi-cassandra-services-nar +Copyright 2016 The Apache Software Foundation + +This product includes software developed at +The Apache Software Foundation (http://www.apache.org/). + +** +Apache Software License v2 +** + +The following binary components are provided under the Apache Software License v2 + + (ASLv2) DataStax Java Driver for Apache Cassandra - Core + The following NOTICE information applies: +DataStax Java Driver for Apache Cassandra - Core +Copyright (C) 2012-2017 DataStax Inc. + + (ASLv2) Apache Avro + The following NOTICE information applies: +Apache Avro +Copyright 2009-2017 The Apache Software Foundation + + (ASLv2) Jackson JSON processor --- End diff -- Understood. > Create a Connection Pooling service implementation to be used in Cassandra > processors > - > > Key: NIFI-5621 > URL: https://issues.apache.org/jira/browse/NIFI-5621 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Sivaprasanna Sethuraman >Assignee: Sivaprasanna Sethuraman >Priority: Major > > Like how the Relational Database processors leverage 'DBCPConnectionPool' > controller service, there should be one that could be used by the processors > from Cassandra bundle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #3105: NIFI-5621: Added Cassandra connection provider serv...
Github user zenfenan commented on a diff in the pull request: https://github.com/apache/nifi/pull/3105#discussion_r239163988 --- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-services-nar/src/main/resources/META-INF/NOTICE --- @@ -0,0 +1,327 @@ +nifi-cassandra-services-nar --- End diff -- Fixed all. Please take a look. ---
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710366#comment-16710366 ] Peter Wicks commented on NIFI-4621: --- Hi [~kislayom], Your PR does not look correct to me. Did you check-in all of the commit's, maybe something got lost? Right now I don't see most of the changes I expected to see (EL changes you mentioned before are missing, etc...) > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Peter Wicks >Priority: Critical > Fix For: 1.9.0 > > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #3203: NIFI-5871 ignore UUID attribute when copying flow f...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3203#discussion_r239160879 --- Diff: nifi-mock/src/main/java/org/apache/nifi/util/MockProcessSession.java --- @@ -496,7 +496,10 @@ public MockFlowFile putAllAttributes(FlowFile flowFile, final Map attrCopy = new HashMap<>(); +attrCopy.putAll(attrs); --- End diff -- Only change I'd make here is putting that into the constructor. Other than that LGTM. ---
[jira] [Commented] (MINIFICPP-689) Make minifi::Exception constructible with string param
[
https://issues.apache.org/jira/browse/MINIFICPP-689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710312#comment-16710312
]
ASF GitHub Bot commented on MINIFICPP-689:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239144665
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
--- End diff --
Ah yah. I don't remember either, but that sounds right If I don't see the
warning when I run the build then all is well that ends well. Thanks.
> Make minifi::Exception constructible with string param
> --
>
> Key: MINIFICPP-689
> URL: https://issues.apache.org/jira/browse/MINIFICPP-689
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Minor
> Fix For: 0.6.0
>
>
> Exception is currently only constructible using const char * argument, but
> that's copied into a string. Using string parameter would make it more
> developer-friendly and save some copy constructions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #455: MINIFICPP-689 - Make minifi::Exception co...
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239145213
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
}
+
// Destructor
- virtual ~Exception() throw () {
+ virtual ~Exception() noexcept {
--- End diff --
No, I wouldn't skip them. Just meant that the change should be identical,
so if it worked, it should still work _in theory_, but we are engineers, so
let's see the results. :)
---
[jira] [Commented] (MINIFICPP-689) Make minifi::Exception constructible with string param
[
https://issues.apache.org/jira/browse/MINIFICPP-689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710314#comment-16710314
]
ASF GitHub Bot commented on MINIFICPP-689:
--
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239145213
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
}
+
// Destructor
- virtual ~Exception() throw () {
+ virtual ~Exception() noexcept {
--- End diff --
No, I wouldn't skip them. Just meant that the change should be identical,
so if it worked, it should still work _in theory_, but we are engineers, so
let's see the results. :)
> Make minifi::Exception constructible with string param
> --
>
> Key: MINIFICPP-689
> URL: https://issues.apache.org/jira/browse/MINIFICPP-689
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Minor
> Fix For: 0.6.0
>
>
> Exception is currently only constructible using const char * argument, but
> that's copied into a string. Using string parameter would make it more
> developer-friendly and save some copy constructions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #455: MINIFICPP-689 - Make minifi::Exception co...
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239144665
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
--- End diff --
Ah yah. I don't remember either, but that sounds right If I don't see the
warning when I run the build then all is well that ends well. Thanks.
---
[GitHub] nifi-minifi-cpp pull request #455: MINIFICPP-689 - Make minifi::Exception co...
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239144107
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
}
+
// Destructor
- virtual ~Exception() throw () {
+ virtual ~Exception() noexcept {
--- End diff --
I'm not sure how that applies to my comment...but I think the end result is
that I will still run basic tests across compilers. Are you suggesting we
needn't run tests for verification and validation?
---
[jira] [Commented] (MINIFICPP-689) Make minifi::Exception constructible with string param
[
https://issues.apache.org/jira/browse/MINIFICPP-689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710307#comment-16710307
]
ASF GitHub Bot commented on MINIFICPP-689:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239144107
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
}
+
// Destructor
- virtual ~Exception() throw () {
+ virtual ~Exception() noexcept {
--- End diff --
I'm not sure how that applies to my comment...but I think the end result is
that I will still run basic tests across compilers. Are you suggesting we
needn't run tests for verification and validation?
> Make minifi::Exception constructible with string param
> --
>
> Key: MINIFICPP-689
> URL: https://issues.apache.org/jira/browse/MINIFICPP-689
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Minor
> Fix For: 0.6.0
>
>
> Exception is currently only constructible using const char * argument, but
> that's copied into a string. Using string parameter would make it more
> developer-friendly and save some copy constructions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-689) Make minifi::Exception constructible with string param
[
https://issues.apache.org/jira/browse/MINIFICPP-689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710295#comment-16710295
]
ASF GitHub Bot commented on MINIFICPP-689:
--
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239140370
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
--- End diff --
As far as I remember that warning is only applied for return values, where
moving could prevent copy-elision, but it's not the case.
> Make minifi::Exception constructible with string param
> --
>
> Key: MINIFICPP-689
> URL: https://issues.apache.org/jira/browse/MINIFICPP-689
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Minor
> Fix For: 0.6.0
>
>
> Exception is currently only constructible using const char * argument, but
> that's copied into a string. Using string parameter would make it more
> developer-friendly and save some copy constructions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #455: MINIFICPP-689 - Make minifi::Exception co...
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239140370
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
--- End diff --
As far as I remember that warning is only applied for return values, where
moving could prevent copy-elision, but it's not the case.
---
[jira] [Commented] (MINIFICPP-689) Make minifi::Exception constructible with string param
[
https://issues.apache.org/jira/browse/MINIFICPP-689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710296#comment-16710296
]
ASF GitHub Bot commented on MINIFICPP-689:
--
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239140994
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
}
+
// Destructor
- virtual ~Exception() throw () {
+ virtual ~Exception() noexcept {
--- End diff --
throw() == noexcept, however throw() is deprecated since C++11.
> Make minifi::Exception constructible with string param
> --
>
> Key: MINIFICPP-689
> URL: https://issues.apache.org/jira/browse/MINIFICPP-689
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Minor
> Fix For: 0.6.0
>
>
> Exception is currently only constructible using const char * argument, but
> that's copied into a string. Using string parameter would make it more
> developer-friendly and save some copy constructions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #455: MINIFICPP-689 - Make minifi::Exception co...
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239140994
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
}
+
// Destructor
- virtual ~Exception() throw () {
+ virtual ~Exception() noexcept {
--- End diff --
throw() == noexcept, however throw() is deprecated since C++11.
---
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710265#comment-16710265 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239125843 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The defailt strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @return 0
[GitHub] nifi-minifi-cpp pull request #448: MINIFICPP-682 - C API: provide functions ...
Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239125843 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The defailt strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @return 0 (success), -1 (strategy cannot be set - no failure callback added?) + **/ int set_failure_strategy(flow *flow, FailureStrategy strategy); -int set_property(processor *, const char *, const char *); +/** + * Set property for a
[GitHub] nifi-minifi-cpp pull request #448: MINIFICPP-682 - C API: provide functions ...
Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239123478 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, --- End diff -- Ok, removed deprecation. ---
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710258#comment-16710258 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239123478 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, --- End diff -- Ok, removed deprecation. > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-689) Make minifi::Exception constructible with string param
[
https://issues.apache.org/jira/browse/MINIFICPP-689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710243#comment-16710243
]
ASF GitHub Bot commented on MINIFICPP-689:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239117109
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
--- End diff --
Just saw this when I was typing the one below, I'll run the build before
merging obviously, but does this not generate an elision warning?
> Make minifi::Exception constructible with string param
> --
>
> Key: MINIFICPP-689
> URL: https://issues.apache.org/jira/browse/MINIFICPP-689
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Minor
> Fix For: 0.6.0
>
>
> Exception is currently only constructible using const char * argument, but
> that's copied into a string. Using string parameter would make it more
> developer-friendly and save some copy constructions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (NIFI-5872) Add compression option on JsonRecordSetWriter
Pierre Villard created NIFI-5872: Summary: Add compression option on JsonRecordSetWriter Key: NIFI-5872 URL: https://issues.apache.org/jira/browse/NIFI-5872 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Pierre Villard Add a compression option on the JsonRecordSetWriter controller service as we have on the AvroRecordSetWriter controller service. This can be useful in case of very IO intensive workflows. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-689) Make minifi::Exception constructible with string param
[
https://issues.apache.org/jira/browse/MINIFICPP-689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710244#comment-16710244
]
ASF GitHub Bot commented on MINIFICPP-689:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239117254
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
}
+
// Destructor
- virtual ~Exception() throw () {
+ virtual ~Exception() noexcept {
--- End diff --
I didn't notice that throw was here. Odd choice. The compiler usually
generates an implicit noexcept unless told otherwise, so I'm tempted to do a
little more testing of this across compilers. I'll do this before merge.
> Make minifi::Exception constructible with string param
> --
>
> Key: MINIFICPP-689
> URL: https://issues.apache.org/jira/browse/MINIFICPP-689
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Minor
> Fix For: 0.6.0
>
>
> Exception is currently only constructible using const char * argument, but
> that's copied into a string. Using string parameter would make it more
> developer-friendly and save some copy constructions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710241#comment-16710241 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239116973 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The defailt strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @return 0
[GitHub] nifi-minifi-cpp pull request #455: MINIFICPP-689 - Make minifi::Exception co...
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239117109
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
--- End diff --
Just saw this when I was typing the one below, I'll run the build before
merging obviously, but does this not generate an elision warning?
---
[GitHub] nifi-minifi-cpp pull request #455: MINIFICPP-689 - Make minifi::Exception co...
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239117254
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
}
+
// Destructor
- virtual ~Exception() throw () {
+ virtual ~Exception() noexcept {
--- End diff --
I didn't notice that throw was here. Odd choice. The compiler usually
generates an implicit noexcept unless told otherwise, so I'm tempted to do a
little more testing of this across compilers. I'll do this before merge.
---
[GitHub] nifi-minifi-cpp pull request #448: MINIFICPP-682 - C API: provide functions ...
Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239116973 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The defailt strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @return 0 (success), -1 (strategy cannot be set - no failure callback added?) + **/ int set_failure_strategy(flow *flow, FailureStrategy strategy); -int set_property(processor *, const char *, const char *); +/** + * Set property for a
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710240#comment-16710240 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239115752 --- Diff: nanofi/include/api/nanofi.h --- @@ -71,62 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The default strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @return 0
[GitHub] nifi-minifi-cpp pull request #448: MINIFICPP-682 - C API: provide functions ...
Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239115752 --- Diff: nanofi/include/api/nanofi.h --- @@ -71,62 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The default strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @return 0 (success), -1 (strategy cannot be set - no failure callback added?) + **/ int set_failure_strategy(flow *flow, FailureStrategy strategy); -int set_property(processor *, const char *, const char *); +/** + * Set property for a
