[jira] [Resolved] (NIFI-5865) Secure Nifi LDAP
[ https://issues.apache.org/jira/browse/NIFI-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-5865. -- Resolution: Information Provided > Secure Nifi LDAP > > > Key: NIFI-5865 > URL: https://issues.apache.org/jira/browse/NIFI-5865 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.8.0 >Reporter: Firdaous hebbal >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5838) Misconfigured Kite processors can block NiFi
[
https://issues.apache.org/jira/browse/NIFI-5838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16709810#comment-16709810
]
Pierre Villard commented on NIFI-5838:
--
I updated the Migration Guidance document to reflect that change. Thanks for
the review Koji!
> Misconfigured Kite processors can block NiFi
>
>
> Key: NIFI-5838
> URL: https://issues.apache.org/jira/browse/NIFI-5838
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Critical
> Fix For: 1.9.0
>
>
> I faced the following situation today: no way to access the NiFi UI (it was
> just hanging forever or throwing timeouts), no change in case of NiFi
> restart, even with disabling the auto resume feature.
> By looking at the thread dump, I found a lot of:
> {noformat}
> "NiFi Web Server-142" Id=142 BLOCKED on
> org.apache.hadoop.ipc.Client$Connection@3843e7a6
> at org.apache.hadoop.ipc.Client$Connection.addCall(Client.java:463)
> at org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:375)
> at org.apache.hadoop.ipc.Client.getConnection(Client.java:1522)
> at org.apache.hadoop.ipc.Client.call(Client.java:1451)
> at org.apache.hadoop.ipc.Client.call(Client.java:1412)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
> at com.sun.proxy.$Proxy348.getBlockLocations(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getBlockLocations(ClientNamenodeProtocolTranslatorPB.java:255)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
> at com.sun.proxy.$Proxy349.getBlockLocations(Unknown Source)
> at
> org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations(DFSClient.java:1226)
> at org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:1213)
> at org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:1201)
> at
> org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:306)
> at org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:272)
> - waiting on java.lang.Object@46d84263
> at org.apache.hadoop.hdfs.DFSInputStream.(DFSInputStream.java:264)
> at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1526)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:304)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:299)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:312)
> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:769)
> at
> org.apache.nifi.processors.kite.AbstractKiteProcessor.getSchema(AbstractKiteProcessor.java:132)
> at
> org.apache.nifi.processors.kite.AbstractKiteProcessor$3.validate(AbstractKiteProcessor.java:172)
> at
> org.apache.nifi.components.PropertyDescriptor.validate(PropertyDescriptor.java:200)
> at
> org.apache.nifi.components.AbstractConfigurableComponent.validate(AbstractConfigurableComponent.java:103)
> at
> org.apache.nifi.controller.AbstractConfiguredComponent.validate(AbstractConfiguredComponent.java:329)
> at
> org.apache.nifi.controller.StandardProcessorNode.isValid(StandardProcessorNode.java:968)
> at
> org.apache.nifi.groups.StandardProcessGroup.getCounts(StandardProcessGroup.java:227)
> at
> org.apache.nifi.groups.StandardProcessGroup.getCounts(StandardProcessGroup.java:261)
> at
> org.apache.nifi.groups.StandardProcessGroup.getCounts(StandardProcessGroup.java:261)
> at
> org.apache.nifi.groups.StandardProcessGroup.getCounts(StandardProcessGroup.java:261)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade.getSiteToSiteDetails(StandardNiFiServiceFacade.java:2609)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
> at
> org.springframework.aop.framework.ReflectiveMethod
[GitHub] nifi pull request #3201: NIFI-4621
GitHub user kislayom opened a pull request: https://github.com/apache/nifi/pull/3201 NIFI-4621 Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [] Have you written or updated unit tests to verify your changes? - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/kislayom/nifi NIFI-4621 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3201.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3201 commit 0512754739cf63785aebfc53a89bb75d37bacbed Author: Kislay Kumar Date: 2018-12-04T12:03:56Z NIFI-4621 Enable ListSFTP and ListFTP processor to accept connection from other processors, also validated the explression language is working for properties of both the processors. commit 9da3a8700244b4fdc06c634805bbc0b25369623a Author: Kislay Kumar Date: 2018-12-04T13:05:34Z NIFI-4621 changes in processor ListSFTP and ListFTP to also have input connection and expression language for properties commit 4319d472a25fecc2bbb66501c98d91828470d15d Author: Kislay Kumar Date: 2018-12-04T12:03:56Z NIFI-4621 Enable ListSFTP and ListFTP processor to accept connection from other processors, also validated the explression language is working for properties of both the processors. NIFI-4621 changes in processor ListSFTP and ListFTP to also have input connection and expression language for properties commit d4321cf72085786f1f0b9453270fc6c7c5d32fe3 Author: kislayom Date: 2018-12-05T09:25:26Z Merge branch 'NIFI-4621' of https://github.com/kislayom/nifi into NIFI-4621 ---
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16709840#comment-16709840 ] ASF GitHub Bot commented on NIFI-4621: -- GitHub user kislayom opened a pull request: https://github.com/apache/nifi/pull/3201 NIFI-4621 Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [] Have you written or updated unit tests to verify your changes? - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/kislayom/nifi NIFI-4621 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3201.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3201 commit 0512754739cf63785aebfc53a89bb75d37bacbed Author: Kislay Kumar Date: 2018-12-04T12:03:56Z NIFI-4621 Enable ListSFTP and ListFTP processor to accept connection from other processors, also validated the explression language is working for properties of both the processors. commit 9da3a8700244b4fdc06c634805bbc0b25369623a Author: Kislay Kumar Date: 2018-12-04T13:05:34Z NIFI-4621 changes in processor ListSFTP and ListFTP to also have input connection and expression language for properties commit 4319d472a25fecc2bbb66501c98d91828470d15d Author: Kislay Kumar Date: 2018-12-04T12:03:56Z NIFI-4621 Enable ListSFTP and ListFTP processor to accept connection from other processors, also validated the explression language is working for properties of both the processors. NIFI-4621 changes in processor ListSFTP and ListFTP to also have input connection and expression language for properties commit d4321cf72085786f1f0b9453270fc6c7c5d32fe3 Author: kislayom Date: 2018-12-05T09:25:26Z Merge branch 'NIFI-4621' of https://github.com/kislayom/nifi into NIFI-4621 > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Kislay Kumar >Priority: Critical > Fix For: 1.9.0 > > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #3201: NIFI-4621
GitHub user kislayom reopened a pull request: https://github.com/apache/nifi/pull/3201 NIFI-4621 Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [] Have you written or updated unit tests to verify your changes? - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/kislayom/nifi NIFI-4621 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3201.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3201 commit 0512754739cf63785aebfc53a89bb75d37bacbed Author: Kislay Kumar Date: 2018-12-04T12:03:56Z NIFI-4621 Enable ListSFTP and ListFTP processor to accept connection from other processors, also validated the explression language is working for properties of both the processors. commit 9da3a8700244b4fdc06c634805bbc0b25369623a Author: Kislay Kumar Date: 2018-12-04T13:05:34Z NIFI-4621 changes in processor ListSFTP and ListFTP to also have input connection and expression language for properties commit 4319d472a25fecc2bbb66501c98d91828470d15d Author: Kislay Kumar Date: 2018-12-04T12:03:56Z NIFI-4621 Enable ListSFTP and ListFTP processor to accept connection from other processors, also validated the explression language is working for properties of both the processors. NIFI-4621 changes in processor ListSFTP and ListFTP to also have input connection and expression language for properties commit d4321cf72085786f1f0b9453270fc6c7c5d32fe3 Author: kislayom Date: 2018-12-05T09:25:26Z Merge branch 'NIFI-4621' of https://github.com/kislayom/nifi into NIFI-4621 ---
[GitHub] nifi pull request #3201: NIFI-4621
Github user kislayom closed the pull request at: https://github.com/apache/nifi/pull/3201 ---
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16709875#comment-16709875 ] ASF GitHub Bot commented on NIFI-4621: -- Github user kislayom closed the pull request at: https://github.com/apache/nifi/pull/3201 > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Kislay Kumar >Priority: Critical > Fix For: 1.9.0 > > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16709876#comment-16709876 ] ASF GitHub Bot commented on NIFI-4621: -- GitHub user kislayom reopened a pull request: https://github.com/apache/nifi/pull/3201 NIFI-4621 Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [] Have you written or updated unit tests to verify your changes? - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/kislayom/nifi NIFI-4621 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3201.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3201 commit 0512754739cf63785aebfc53a89bb75d37bacbed Author: Kislay Kumar Date: 2018-12-04T12:03:56Z NIFI-4621 Enable ListSFTP and ListFTP processor to accept connection from other processors, also validated the explression language is working for properties of both the processors. commit 9da3a8700244b4fdc06c634805bbc0b25369623a Author: Kislay Kumar Date: 2018-12-04T13:05:34Z NIFI-4621 changes in processor ListSFTP and ListFTP to also have input connection and expression language for properties commit 4319d472a25fecc2bbb66501c98d91828470d15d Author: Kislay Kumar Date: 2018-12-04T12:03:56Z NIFI-4621 Enable ListSFTP and ListFTP processor to accept connection from other processors, also validated the explression language is working for properties of both the processors. NIFI-4621 changes in processor ListSFTP and ListFTP to also have input connection and expression language for properties commit d4321cf72085786f1f0b9453270fc6c7c5d32fe3 Author: kislayom Date: 2018-12-05T09:25:26Z Merge branch 'NIFI-4621' of https://github.com/kislayom/nifi into NIFI-4621 > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Kislay Kumar >Priority: Critical > Fix For: 1.9.0 > > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #3199: NIFI-5863 Fixed AbstractFlowFileQueue logging
Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/3199 +1, merged to master, thanks @ijokarumawak ---
[GitHub] nifi pull request #3199: NIFI-5863 Fixed AbstractFlowFileQueue logging
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/3199 ---
[jira] [Commented] (NIFI-5863) Invalid logging in AbstractFlowFileQueue.java
[
https://issues.apache.org/jira/browse/NIFI-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16709890#comment-16709890
]
ASF GitHub Bot commented on NIFI-5863:
--
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/3199
> Invalid logging in AbstractFlowFileQueue.java
> -
>
> Key: NIFI-5863
> URL: https://issues.apache.org/jira/browse/NIFI-5863
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.8.0
>Reporter: Truong Duc Kien
>Assignee: Koji Kawamura
>Priority: Minor
>
> The variable \{{resultCount}} is never updated, so the logging message always
> report 0 results.
> https://github.com/apache/nifi/blob/234ddb0fe1a36ad947c340114058d82c777d791f/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/queue/AbstractFlowFileQueue.java#L219
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-5863) Invalid logging in AbstractFlowFileQueue.java
[
https://issues.apache.org/jira/browse/NIFI-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-5863:
-
Resolution: Fixed
Fix Version/s: 1.9.0
Status: Resolved (was: Patch Available)
> Invalid logging in AbstractFlowFileQueue.java
> -
>
> Key: NIFI-5863
> URL: https://issues.apache.org/jira/browse/NIFI-5863
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.8.0
>Reporter: Truong Duc Kien
>Assignee: Koji Kawamura
>Priority: Minor
> Fix For: 1.9.0
>
>
> The variable \{{resultCount}} is never updated, so the logging message always
> report 0 results.
> https://github.com/apache/nifi/blob/234ddb0fe1a36ad947c340114058d82c777d791f/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/queue/AbstractFlowFileQueue.java#L219
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5863) Invalid logging in AbstractFlowFileQueue.java
[
https://issues.apache.org/jira/browse/NIFI-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16709889#comment-16709889
]
ASF GitHub Bot commented on NIFI-5863:
--
Github user pvillard31 commented on the issue:
https://github.com/apache/nifi/pull/3199
+1, merged to master, thanks @ijokarumawak
> Invalid logging in AbstractFlowFileQueue.java
> -
>
> Key: NIFI-5863
> URL: https://issues.apache.org/jira/browse/NIFI-5863
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.8.0
>Reporter: Truong Duc Kien
>Assignee: Koji Kawamura
>Priority: Minor
>
> The variable \{{resultCount}} is never updated, so the logging message always
> report 0 results.
> https://github.com/apache/nifi/blob/234ddb0fe1a36ad947c340114058d82c777d791f/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/queue/AbstractFlowFileQueue.java#L219
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5863) Invalid logging in AbstractFlowFileQueue.java
[
https://issues.apache.org/jira/browse/NIFI-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16709888#comment-16709888
]
ASF subversion and git services commented on NIFI-5863:
---
Commit 8ebb4d19740a3ccb879c03b322b335d02b6b16dc in nifi's branch
refs/heads/master from [~ijokarumawak]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=8ebb4d1 ]
NIFI-5863 Fixed AbstractFlowFileQueue logging
Signed-off-by: Pierre Villard
This closes #3199.
> Invalid logging in AbstractFlowFileQueue.java
> -
>
> Key: NIFI-5863
> URL: https://issues.apache.org/jira/browse/NIFI-5863
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.8.0
>Reporter: Truong Duc Kien
>Assignee: Koji Kawamura
>Priority: Minor
>
> The variable \{{resultCount}} is never updated, so the logging message always
> report 0 results.
> https://github.com/apache/nifi/blob/234ddb0fe1a36ad947c340114058d82c777d791f/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/queue/AbstractFlowFileQueue.java#L219
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #3198: NIFI-5867 - add thread() EL function to get thread ...
Github user pvillard31 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3198#discussion_r239017060
--- Diff:
nifi-commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/ThreadEvaluator.java
---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.attribute.expression.language.evaluation.functions;
+
+import java.util.Map;
+
+import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
+import
org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
+import
org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator;
+import
org.apache.nifi.attribute.expression.language.evaluation.StringQueryResult;
+
+public class ThreadEvaluator extends StringEvaluator {
+
+private final StringQueryResult thread;
+
+public ThreadEvaluator() {
+// See org.apache.nifi.engine.FlowEngine
+thread = new StringQueryResult(Thread.currentThread().getName());
--- End diff --
yeah... good point, pushed a commit to address that. Thanks!
---
[jira] [Commented] (NIFI-5867) Add EL function to get thread name
[
https://issues.apache.org/jira/browse/NIFI-5867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16709909#comment-16709909
]
ASF GitHub Bot commented on NIFI-5867:
--
Github user pvillard31 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3198#discussion_r239017060
--- Diff:
nifi-commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/ThreadEvaluator.java
---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.attribute.expression.language.evaluation.functions;
+
+import java.util.Map;
+
+import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
+import
org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
+import
org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator;
+import
org.apache.nifi.attribute.expression.language.evaluation.StringQueryResult;
+
+public class ThreadEvaluator extends StringEvaluator {
+
+private final StringQueryResult thread;
+
+public ThreadEvaluator() {
+// See org.apache.nifi.engine.FlowEngine
+thread = new StringQueryResult(Thread.currentThread().getName());
--- End diff --
yeah... good point, pushed a commit to address that. Thanks!
> Add EL function to get thread name
> --
>
> Key: NIFI-5867
> URL: https://issues.apache.org/jira/browse/NIFI-5867
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Major
>
> Add an Expression Language function to get the thread name doing the
> expression evaluation. This can be useful in case a processor is configured
> with multiple concurrent tasks and requires some data uniqueness.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (MINIFICPP-689) Make minifi::Exception constructible with string param
Arpad Boda created MINIFICPP-689: Summary: Make minifi::Exception constructible with string param Key: MINIFICPP-689 URL: https://issues.apache.org/jira/browse/MINIFICPP-689 Project: NiFi MiNiFi C++ Issue Type: Improvement Reporter: Arpad Boda Assignee: Arpad Boda Fix For: 0.6.0 Exception is currently only constructible using const char * argument, but that's copied into a string. Using string parameter would make it more developer-friendly and save some copy constructions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #455: MINIFICPP-689 - Make minifi::Exception co...
GitHub user arpadboda opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/455 MINIFICPP-689 - Make minifi::Exception constructible with string param Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/arpadboda/nifi-minifi-cpp MINIFICPP-689 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/455.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #455 commit 7238bfa77e587f6bb370ea7733431961fb145a5f Author: Arpad Boda Date: 2018-12-05T11:45:42Z MINIFICPP-689 - Make minifi::Exception constructible with string param ---
[jira] [Commented] (MINIFICPP-689) Make minifi::Exception constructible with string param
[ https://issues.apache.org/jira/browse/MINIFICPP-689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16709965#comment-16709965 ] ASF GitHub Bot commented on MINIFICPP-689: -- GitHub user arpadboda opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/455 MINIFICPP-689 - Make minifi::Exception constructible with string param Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/arpadboda/nifi-minifi-cpp MINIFICPP-689 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/455.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #455 commit 7238bfa77e587f6bb370ea7733431961fb145a5f Author: Arpad Boda Date: 2018-12-05T11:45:42Z MINIFICPP-689 - Make minifi::Exception constructible with string param > Make minifi::Exception constructible with string param > -- > > Key: MINIFICPP-689 > URL: https://issues.apache.org/jira/browse/MINIFICPP-689 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Minor > Fix For: 0.6.0 > > > Exception is currently only constructible using const char * argument, but > that's copied into a string. Using string parameter would make it more > developer-friendly and save some copy constructions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-5870) Possible race condition when receiving flow files from peers during load balancing.
Truong Duc Kien created NIFI-5870:
-
Summary: Possible race condition when receiving flow files from
peers during load balancing.
Key: NIFI-5870
URL: https://issues.apache.org/jira/browse/NIFI-5870
Project: Apache NiFi
Issue Type: Bug
Reporter: Truong Duc Kien
The queue size should always be adjusted before FlowFiles are added into the
local partition, as explained in the method \{{putAllAndGetPartitions}} .
However, the method \{{receiveFromPeer}} is doing it backward.
https://github.com/apache/nifi/blob/234ddb0fe1a36ad947c340114058d82c777d791f/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/queue/clustered/SocketLoadBalancedFlowFileQueue.java#L772
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kislay Kumar resolved NIFI-4621. Resolution: Fixed [~ijokarumawak]: Thanks Koji for the help. [~patricker] [~ijokarumawak] Please review the PR. Let me know your review points. Kind Regards Kislay > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Kislay Kumar >Priority: Critical > Fix For: 1.9.0 > > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kislay Kumar reassigned NIFI-4621: -- Assignee: Peter Wicks (was: Kislay Kumar) > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Peter Wicks >Priority: Critical > Fix For: 1.9.0 > > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5639) PublishAMQP Processor
[ https://issues.apache.org/jira/browse/NIFI-5639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710006#comment-16710006 ] Kislay Kumar commented on NIFI-5639: [~patricker], [~ijokarumawak] : Please advice if this is a good Jira to pick. > PublishAMQP Processor > --- > > Key: NIFI-5639 > URL: https://issues.apache.org/jira/browse/NIFI-5639 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.5.0, 1.6.0, 1.7.0, 1.7.1 >Reporter: Mohammed Nadeem >Priority: Critical > Original Estimate: 96h > Remaining Estimate: 96h > > *PublishAMQP* Processor is routing incoming flowfile to success relationship > though invalid properties are configured. The processor is not throwing any > errors when invalid properties are configured at run-time. When the message > is not published with invalid properties, the processor silently routes the > message to success when it is suppose to route to failure relationship > indicating message was not published to the queue. This is a bug and no > proper error handling when failures occur *(nifi-amqp-bundle)* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5851) Add MFA to *S3Object Processors
[ https://issues.apache.org/jira/browse/NIFI-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710112#comment-16710112 ] Sivaprasanna Sethuraman commented on NIFI-5851: --- [~LH] I think the link is broken. Can you please share it again? > Add MFA to *S3Object Processors > --- > > Key: NIFI-5851 > URL: https://issues.apache.org/jira/browse/NIFI-5851 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Lukas Heusser >Priority: Major > Labels: features, security > > Add the possibility to add MFA to the *S3Object processors. Due to security > this is a quiet important feature for customers. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #454: MINIFICPP-404: Correct invalid assumption...
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/454#discussion_r239092725
--- Diff: libminifi/src/core/yaml/YamlConfiguration.cpp ---
@@ -325,6 +325,12 @@ void
YamlConfiguration::parseRemoteProcessGroupYaml(YAML::Node *rpgNode, core::P
}
}
}
+ } else if (transport_protocol == "RAW") {
+group->setTransportProtocol(transport_protocol);
+ } else {
+std::stringstream stream;
+stream << "Invalid transport protocol " << transport_protocol;
+throw minifi::Exception(ExceptionType::SITE2SITE_EXCEPTION,
stream.str().c_str());
--- End diff --
I think it's vestigial (existed since the early days of the project )-- no
reason, just no desire to make changes that require unnecessary classes to
change in this PR. If someone said, "it has to be done to approve," I'd go
ahead and do it but a follow on PR like the one you made is perfect.
---
[jira] [Commented] (MINIFICPP-404) HTTP Proxy Support for HTTP Site to Site
[
https://issues.apache.org/jira/browse/MINIFICPP-404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710138#comment-16710138
]
ASF GitHub Bot commented on MINIFICPP-404:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/454#discussion_r239092725
--- Diff: libminifi/src/core/yaml/YamlConfiguration.cpp ---
@@ -325,6 +325,12 @@ void
YamlConfiguration::parseRemoteProcessGroupYaml(YAML::Node *rpgNode, core::P
}
}
}
+ } else if (transport_protocol == "RAW") {
+group->setTransportProtocol(transport_protocol);
+ } else {
+std::stringstream stream;
+stream << "Invalid transport protocol " << transport_protocol;
+throw minifi::Exception(ExceptionType::SITE2SITE_EXCEPTION,
stream.str().c_str());
--- End diff --
I think it's vestigial (existed since the early days of the project )-- no
reason, just no desire to make changes that require unnecessary classes to
change in this PR. If someone said, "it has to be done to approve," I'd go
ahead and do it but a follow on PR like the one you made is perfect.
> HTTP Proxy Support for HTTP Site to Site
>
>
> Key: MINIFICPP-404
> URL: https://issues.apache.org/jira/browse/MINIFICPP-404
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Affects Versions: 0.6.0
>Reporter: bqiu
>Assignee: Mr TheSegfault
>Priority: Minor
> Fix For: 0.6.0
>
>
> HTTP Proxy Support for HTTP Site to Site
> http://nifi.apache.org/docs/nifi-docs/html/user-guide.html#configure-site-to-site-client-nifi-instance.
> support for this in YAML config via
> https://github.com/apache/nifi-minifi/blob/master/minifi-docs/src/main/markdown/System_Admin_Guide.md#remote-process-groups-1.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (NIFI-5871) MockProcessSession.putAllAttributes should ignore the UUID attribute
Alex Savitsky created NIFI-5871: --- Summary: MockProcessSession.putAllAttributes should ignore the UUID attribute Key: NIFI-5871 URL: https://issues.apache.org/jira/browse/NIFI-5871 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.8.0 Reporter: Alex Savitsky Currently, the method would copy all attributes indiscriminately, but the interface Javadoc specifically states that the attribute "uuid" should be ignored. This leads to issues with testing, where two distinct flow files are considered same in the session. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5851) Add MFA to *S3Object Processors
[ https://issues.apache.org/jira/browse/NIFI-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710156#comment-16710156 ] Lukas Heusser commented on NIFI-5851: - [~sivaprasanna] Sorry my bad. Here you go [https://aws.amazon.com/de/premiumsupport/knowledge-center/authenticate-mfa-cli/] > Add MFA to *S3Object Processors > --- > > Key: NIFI-5851 > URL: https://issues.apache.org/jira/browse/NIFI-5851 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Lukas Heusser >Priority: Major > Labels: features, security > > Add the possibility to add MFA to the *S3Object processors. Due to security > this is a quiet important feature for customers. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #3202: NIFI-5868: Added instrumentation around ListFile su...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/3202 NIFI-5868: Added instrumentation around ListFile such that all disk a⦠â¦ccesses are timed and any unusually long listing times or disk access operations can be logged. Additionally, information is logged at a debug level including significant amounts of troubleshooting information when configured to do so Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi nIFI-5868 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3202.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3202 commit 49cc37d807c52701fcce7cae0c1a2265ead0 Author: Mark Payne Date: 2018-12-05T14:37:20Z NIFI-5868: Added instrumentation around ListFile such that all disk accesses are timed and any unusually long listing times or disk access operations can be logged. Additionally, information is logged at a debug level including significant amounts of troubleshooting information when configured to do so ---
[jira] [Updated] (NIFI-5868) Instrument robust timing information for ListFile
[ https://issues.apache.org/jira/browse/NIFI-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-5868: - Fix Version/s: 1.9.0 Status: Patch Available (was: Open) > Instrument robust timing information for ListFile > - > > Key: NIFI-5868 > URL: https://issues.apache.org/jira/browse/NIFI-5868 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.9.0 > > > ListFile is used in many different contexts. We often see users with a > specific use case, though, which is to run ListFile on a Primary Node in a > cluster, in order to obtain a file listing of an NFS-mounted share. This > works well in most cases, but whenever problems do arise, it is very > difficult to understand what the problem is. It would be very helpful to have > information such as: > * Is there a problem accessing a specific file on the NFS mount? > * Is there a problem obtaining a listing from the NFS mount? > * Is progress being made at all? > * How long is a listing taking right now? > * How long does a listing typically take? > * Is this problem related to NiFi or to the operating system / > infrastructure? > It would be helpful to track information about each disk access that is > occurring, as well as the overall listing progress and issue warnings if we > see clear problems arise. We can do this by timing how long each disk access > takes, what file was being accessed, and what operating was being performed. > If we capture this data in a rolling window, we can assess the data to > determine if the listing is now taking longer than it did previously and > alert to this fact. Or alert if performing a specific disk operation is > taking a long time. > Gathering this information will likely be fairly heap intensive, so it is > best to make the functionality optional. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5868) Instrument robust timing information for ListFile
[ https://issues.apache.org/jira/browse/NIFI-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710162#comment-16710162 ] ASF GitHub Bot commented on NIFI-5868: -- GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/3202 NIFI-5868: Added instrumentation around ListFile such that all disk a… …ccesses are timed and any unusually long listing times or disk access operations can be logged. Additionally, information is logged at a debug level including significant amounts of troubleshooting information when configured to do so Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi nIFI-5868 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3202.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3202 commit 49cc37d807c52701fcce7cae0c1a2265ead0 Author: Mark Payne Date: 2018-12-05T14:37:20Z NIFI-5868: Added instrumentation around ListFile such that all disk accesses are timed and any unusually long listing times or disk access operations can be logged. Additionally, information is logged at a debug level including significant amounts of troubleshooting information when configured to do so > Instrument robust timing information for ListFile > - > > Key: NIFI-5868 > URL: https://issues.apache.org/jira/browse/NIFI-5868 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.9.0 > > > ListFile is used in many different contexts. We often see users with a > specific use case, though, which is to run ListFile on a Primary Node in a > cluster, in order to obtain a file listing of an NFS-mounted share. This > works well in most cases, but whenever problems do arise, it is very > difficult to understand what the problem is. It would be very helpful to have > information such as: > * Is there a problem accessing a specific file on the NFS mount? > * Is there a problem obtaining a listing from the NFS mount? > * Is progress being made at all? > * How long is a listing taking right now? > * How long does a listing typically take? > * Is this problem related to NiFi or to the operating system / > infrastructure? > It would be helpful to track information about each disk access that is > occurring, as well as the overall listing progress and issue warnings if we > see clear problems arise. We can do this by timing how long each disk access > takes, what file was being accessed, and what operating was being performed. > If we capture this data in a rolling window, we can assess the data to > determine if the listing is now taking longer than it did previously and > alert to this fact. Or alert if performing a specific disk opera
[GitHub] nifi-minifi-cpp issue #448: MINIFICPP-682 - C API: provide functions to crea...
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/448 taking a look ---
[jira] [Commented] (NIFI-5859) Update NAR maven plugin to include information about Extensions
[
https://issues.apache.org/jira/browse/NIFI-5859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710175#comment-16710175
]
ASF GitHub Bot commented on NIFI-5859:
--
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3192#discussion_r239101308
--- Diff:
nifi-api/src/main/java/org/apache/nifi/documentation/AbstractDocumentationWriter.java
---
@@ -0,0 +1,301 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.documentation;
+
+import org.apache.nifi.annotation.behavior.DynamicProperties;
+import org.apache.nifi.annotation.behavior.DynamicProperty;
+import org.apache.nifi.annotation.behavior.DynamicRelationship;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.ReadsAttribute;
+import org.apache.nifi.annotation.behavior.ReadsAttributes;
+import org.apache.nifi.annotation.behavior.Restricted;
+import org.apache.nifi.annotation.behavior.Stateful;
+import org.apache.nifi.annotation.behavior.SystemResourceConsideration;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.DeprecationNotice;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.ConfigurableComponent;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.ControllerService;
+import
org.apache.nifi.documentation.init.DocumentationControllerServiceInitializationContext;
+import
org.apache.nifi.documentation.init.DocumentationProcessorInitializationContext;
+import
org.apache.nifi.documentation.init.DocumentationReportingInitializationContext;
+import org.apache.nifi.processor.Processor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.reporting.ReportingTask;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.io.Reader;
+import java.io.StringWriter;
+import java.io.Writer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.List;
+import java.util.Set;
+
+/**
+ * Base class for DocumentationWriter that simplifies iterating over all
information for a component, creating a separate method
+ * for each, to ensure that implementations properly override all methods
and therefore properly account for all information about
+ * a component.
+ *
+ * Please note that while this class lives within the nifi-api, it is
provided primarily as a means for documentation components within
+ * the NiFi NAR Maven Plugin. Its home is the nifi-api, however, because
the API is needed in order to extract the relevant information and
+ * the NAR Maven Plugin cannot have a direct dependency on nifi-api (doing
so would cause a circular dependency). By having this homed within
+ * the nifi-api, the Maven plugin is able to discover the class
dynamically and invoke the one or two methods necessary to create the
documentation.
+ *
+ * This is a new capability in 1.9.0 in preparation for the Extension
Registry and therefore, you should
+ * NOTE WELL: At this time, while this class is part of nifi-api,
it is still evolving and may change in a non-backward-compatible manner or even
be
+ * removed from one incremental release to the next. Use at your own risk!
+ */
+public abstract class AbstractDocumentationWriter implements
DocumentationWriter {
+
+@Override
+public final void write(final ConfigurableComponent compone
[GitHub] nifi pull request #3192: NIFI-5859: Added XML-based documentation writer tha...
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3192#discussion_r239101308
--- Diff:
nifi-api/src/main/java/org/apache/nifi/documentation/AbstractDocumentationWriter.java
---
@@ -0,0 +1,301 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.documentation;
+
+import org.apache.nifi.annotation.behavior.DynamicProperties;
+import org.apache.nifi.annotation.behavior.DynamicProperty;
+import org.apache.nifi.annotation.behavior.DynamicRelationship;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.ReadsAttribute;
+import org.apache.nifi.annotation.behavior.ReadsAttributes;
+import org.apache.nifi.annotation.behavior.Restricted;
+import org.apache.nifi.annotation.behavior.Stateful;
+import org.apache.nifi.annotation.behavior.SystemResourceConsideration;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.DeprecationNotice;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.ConfigurableComponent;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.ControllerService;
+import
org.apache.nifi.documentation.init.DocumentationControllerServiceInitializationContext;
+import
org.apache.nifi.documentation.init.DocumentationProcessorInitializationContext;
+import
org.apache.nifi.documentation.init.DocumentationReportingInitializationContext;
+import org.apache.nifi.processor.Processor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.reporting.ReportingTask;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.io.Reader;
+import java.io.StringWriter;
+import java.io.Writer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.List;
+import java.util.Set;
+
+/**
+ * Base class for DocumentationWriter that simplifies iterating over all
information for a component, creating a separate method
+ * for each, to ensure that implementations properly override all methods
and therefore properly account for all information about
+ * a component.
+ *
+ * Please note that while this class lives within the nifi-api, it is
provided primarily as a means for documentation components within
+ * the NiFi NAR Maven Plugin. Its home is the nifi-api, however, because
the API is needed in order to extract the relevant information and
+ * the NAR Maven Plugin cannot have a direct dependency on nifi-api (doing
so would cause a circular dependency). By having this homed within
+ * the nifi-api, the Maven plugin is able to discover the class
dynamically and invoke the one or two methods necessary to create the
documentation.
+ *
+ * This is a new capability in 1.9.0 in preparation for the Extension
Registry and therefore, you should
+ * NOTE WELL: At this time, while this class is part of nifi-api,
it is still evolving and may change in a non-backward-compatible manner or even
be
+ * removed from one incremental release to the next. Use at your own risk!
+ */
+public abstract class AbstractDocumentationWriter implements
DocumentationWriter {
+
+@Override
+public final void write(final ConfigurableComponent component) throws
IOException {
+write(component, null);
+}
+
+@Override
+public final void write(final ConfigurableComponent component, final
Collection providedServices) throws IOException {
+initialize(co
[GitHub] nifi pull request #3192: NIFI-5859: Added XML-based documentation writer tha...
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3192#discussion_r239102517
--- Diff:
nifi-api/src/main/java/org/apache/nifi/documentation/AbstractDocumentationWriter.java
---
@@ -0,0 +1,301 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.documentation;
+
+import org.apache.nifi.annotation.behavior.DynamicProperties;
+import org.apache.nifi.annotation.behavior.DynamicProperty;
+import org.apache.nifi.annotation.behavior.DynamicRelationship;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.ReadsAttribute;
+import org.apache.nifi.annotation.behavior.ReadsAttributes;
+import org.apache.nifi.annotation.behavior.Restricted;
+import org.apache.nifi.annotation.behavior.Stateful;
+import org.apache.nifi.annotation.behavior.SystemResourceConsideration;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.DeprecationNotice;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.ConfigurableComponent;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.ControllerService;
+import
org.apache.nifi.documentation.init.DocumentationControllerServiceInitializationContext;
+import
org.apache.nifi.documentation.init.DocumentationProcessorInitializationContext;
+import
org.apache.nifi.documentation.init.DocumentationReportingInitializationContext;
+import org.apache.nifi.processor.Processor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.reporting.ReportingTask;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.io.Reader;
+import java.io.StringWriter;
+import java.io.Writer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.List;
+import java.util.Set;
+
+/**
+ * Base class for DocumentationWriter that simplifies iterating over all
information for a component, creating a separate method
+ * for each, to ensure that implementations properly override all methods
and therefore properly account for all information about
+ * a component.
+ *
+ * Please note that while this class lives within the nifi-api, it is
provided primarily as a means for documentation components within
+ * the NiFi NAR Maven Plugin. Its home is the nifi-api, however, because
the API is needed in order to extract the relevant information and
+ * the NAR Maven Plugin cannot have a direct dependency on nifi-api (doing
so would cause a circular dependency). By having this homed within
+ * the nifi-api, the Maven plugin is able to discover the class
dynamically and invoke the one or two methods necessary to create the
documentation.
+ *
+ * This is a new capability in 1.9.0 in preparation for the Extension
Registry and therefore, you should
+ * NOTE WELL: At this time, while this class is part of nifi-api,
it is still evolving and may change in a non-backward-compatible manner or even
be
+ * removed from one incremental release to the next. Use at your own risk!
+ */
+public abstract class AbstractDocumentationWriter implements
DocumentationWriter {
+
+@Override
+public final void write(final ConfigurableComponent component) throws
IOException {
+write(component, null);
+}
+
+@Override
+public final void write(final ConfigurableComponent component, final
Collection providedServices) throws IOException {
+initialize(co
[jira] [Commented] (NIFI-5859) Update NAR maven plugin to include information about Extensions
[ https://issues.apache.org/jira/browse/NIFI-5859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710179#comment-16710179 ] ASF GitHub Bot commented on NIFI-5859: -- Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/3192#discussion_r239102417 --- Diff: nifi-nar-bundles/nifi-standard-services/nifi-standard-services-api-nar/pom.xml --- @@ -26,6 +26,12 @@ true + +org.apache.nifi +nifi-jetty-bundle +1.9.0-SNAPSHOT +nar --- End diff -- It is needed if any component has a Custom UI. Otherwise, the NAR plugin is unable to generate the documentation needed. While NiFi will create the linkage for you, it can do so because it knows that the NAR is in the "lib" directory. Without this, the plugin doesn't know what version of the jetty bundle to use. This will be necessary for any component that contains a Custom UI. It is, however, considered a best practice already to include this in such a case, because you are explicitly depending on those classes being in the classpath, so you should explicitly declare it. > Update NAR maven plugin to include information about Extensions > --- > > Key: NIFI-5859 > URL: https://issues.apache.org/jira/browse/NIFI-5859 > Project: Apache NiFi > Issue Type: New Feature > Components: Tools and Build >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > > In order to have the NiFi Registry host any extensions, the registry will > need a way to know what extensions exist in a given NAR. Currently, that > information is not available directly. > The NAR maven plugin should be updated to provide a list of extensions and > for each one, provide at least the following minimal information: > * Extension Type > * Extension Name > * Capability Description > * Whether or not the component is Restricted, "sub-restrictions" it has, and > explanations of both > * Any Tags that the component has > * If the component is a Controller Service, any Controller Service API's > that it provides > Additionally, it would be ideal to provide all documentation for the > component within the NAR. It is best, though, not to write the documentation > in HTML as is done now but rather in XML or some sort of form that provides > the information in a structured way without any styling. This would allow the > documentation to be rendered consistently, even if the styling changes from 1 > version to the next. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #3202: NIFI-5868: Added instrumentation around ListFile such that...
Github user bbende commented on the issue: https://github.com/apache/nifi/pull/3202 Will review... ---
[jira] [Commented] (NIFI-5859) Update NAR maven plugin to include information about Extensions
[
https://issues.apache.org/jira/browse/NIFI-5859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710181#comment-16710181
]
ASF GitHub Bot commented on NIFI-5859:
--
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3192#discussion_r239102517
--- Diff:
nifi-api/src/main/java/org/apache/nifi/documentation/AbstractDocumentationWriter.java
---
@@ -0,0 +1,301 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.documentation;
+
+import org.apache.nifi.annotation.behavior.DynamicProperties;
+import org.apache.nifi.annotation.behavior.DynamicProperty;
+import org.apache.nifi.annotation.behavior.DynamicRelationship;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.ReadsAttribute;
+import org.apache.nifi.annotation.behavior.ReadsAttributes;
+import org.apache.nifi.annotation.behavior.Restricted;
+import org.apache.nifi.annotation.behavior.Stateful;
+import org.apache.nifi.annotation.behavior.SystemResourceConsideration;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.DeprecationNotice;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.ConfigurableComponent;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.ControllerService;
+import
org.apache.nifi.documentation.init.DocumentationControllerServiceInitializationContext;
+import
org.apache.nifi.documentation.init.DocumentationProcessorInitializationContext;
+import
org.apache.nifi.documentation.init.DocumentationReportingInitializationContext;
+import org.apache.nifi.processor.Processor;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.reporting.ReportingTask;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.io.Reader;
+import java.io.StringWriter;
+import java.io.Writer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.List;
+import java.util.Set;
+
+/**
+ * Base class for DocumentationWriter that simplifies iterating over all
information for a component, creating a separate method
+ * for each, to ensure that implementations properly override all methods
and therefore properly account for all information about
+ * a component.
+ *
+ * Please note that while this class lives within the nifi-api, it is
provided primarily as a means for documentation components within
+ * the NiFi NAR Maven Plugin. Its home is the nifi-api, however, because
the API is needed in order to extract the relevant information and
+ * the NAR Maven Plugin cannot have a direct dependency on nifi-api (doing
so would cause a circular dependency). By having this homed within
+ * the nifi-api, the Maven plugin is able to discover the class
dynamically and invoke the one or two methods necessary to create the
documentation.
+ *
+ * This is a new capability in 1.9.0 in preparation for the Extension
Registry and therefore, you should
+ * NOTE WELL: At this time, while this class is part of nifi-api,
it is still evolving and may change in a non-backward-compatible manner or even
be
+ * removed from one incremental release to the next. Use at your own risk!
+ */
+public abstract class AbstractDocumentationWriter implements
DocumentationWriter {
+
+@Override
+public final void write(final ConfigurableComponent compone
[GitHub] nifi pull request #3192: NIFI-5859: Added XML-based documentation writer tha...
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/3192#discussion_r239102417 --- Diff: nifi-nar-bundles/nifi-standard-services/nifi-standard-services-api-nar/pom.xml --- @@ -26,6 +26,12 @@ true + +org.apache.nifi +nifi-jetty-bundle +1.9.0-SNAPSHOT +nar --- End diff -- It is needed if any component has a Custom UI. Otherwise, the NAR plugin is unable to generate the documentation needed. While NiFi will create the linkage for you, it can do so because it knows that the NAR is in the "lib" directory. Without this, the plugin doesn't know what version of the jetty bundle to use. This will be necessary for any component that contains a Custom UI. It is, however, considered a best practice already to include this in such a case, because you are explicitly depending on those classes being in the classpath, so you should explicitly declare it. ---
[GitHub] nifi-minifi-cpp pull request #448: MINIFICPP-682 - C API: provide functions ...
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239104013 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, --- End diff -- This is precisely something we want to keep by design. Let's talk about this before merging this. ---
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710191#comment-16710191 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239104013 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, --- End diff -- This is precisely something we want to keep by design. Let's talk about this before merging this. > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #448: MINIFICPP-682 - C API: provide functions ...
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239103770 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, --- End diff -- I'm unclear about deprecation. Does this mean that it's deprecated because you have no way of indicating the processor could not be instantiated? Wouldn't that arrive at the case where flow is null? Flow in create_new_flow could be null by virtue of a malloc error, so why deprecate a function that results in the same behavior and fewer function calls? ---
[GitHub] nifi-minifi-cpp pull request #448: MINIFICPP-682 - C API: provide functions ...
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239105138 --- Diff: nanofi/include/api/nanofi.h --- @@ -71,62 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); --- End diff -- This was starved off due to other work, but we will eventually need directory specific accessor functions for languages , I would imagine. This could be a candidate for that. ---
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710170#comment-16710170 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/448 taking a look > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710189#comment-16710189 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239103770 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, --- End diff -- I'm unclear about deprecation. Does this mean that it's deprecated because you have no way of indicating the processor could not be instantiated? Wouldn't that arrive at the case where flow is null? Flow in create_new_flow could be null by virtue of a malloc error, so why deprecate a function that results in the same behavior and fewer function calls? > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710200#comment-16710200 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239105138 --- Diff: nanofi/include/api/nanofi.h --- @@ -71,62 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); --- End diff -- This was starved off due to other work, but we will eventually need directory specific accessor functions for languages , I would imagine. This could be a candidate for that. > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #3203: NIFI-5871 ignore UUID attribute when copying flow f...
GitHub user SavtechSolutions opened a pull request: https://github.com/apache/nifi/pull/3203 NIFI-5871 ignore UUID attribute when copying flow file attributes Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/SavtechSolutions/nifi NIFI-5871 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3203.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3203 commit 932c4efa211d21c0a9e2c0e402ce9bd0b2bfc5a3 Author: Alex Savitsky Date: 2018-12-05T14:59:42Z NIFI-5871 ignore UUID attribute when copying flow file attributes ---
[jira] [Commented] (NIFI-5871) MockProcessSession.putAllAttributes should ignore the UUID attribute
[ https://issues.apache.org/jira/browse/NIFI-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710207#comment-16710207 ] ASF GitHub Bot commented on NIFI-5871: -- GitHub user SavtechSolutions opened a pull request: https://github.com/apache/nifi/pull/3203 NIFI-5871 ignore UUID attribute when copying flow file attributes Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/SavtechSolutions/nifi NIFI-5871 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3203.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3203 commit 932c4efa211d21c0a9e2c0e402ce9bd0b2bfc5a3 Author: Alex Savitsky Date: 2018-12-05T14:59:42Z NIFI-5871 ignore UUID attribute when copying flow file attributes > MockProcessSession.putAllAttributes should ignore the UUID attribute > > > Key: NIFI-5871 > URL: https://issues.apache.org/jira/browse/NIFI-5871 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.8.0 >Reporter: Alex Savitsky >Priority: Major > > Currently, the method would copy all attributes indiscriminately, but the > interface Javadoc specifically states that the attribute "uuid" should be > ignored. This leads to issues with testing, where two distinct flow files are > considered same in the session. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710204#comment-16710204 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239105384 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, --- End diff -- The problematic case is flow being instantiated properly, but the processor doesn't (the name is wrong for eg.). In this case a valid flow pointer is returned, but the flow doesn't contain any processor. To make it worse: -There is no function in the API to get (at least the number) of processors in the flow -As a valid ptr was returned, the caller could expect everything to be fine, but that's not the case. > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #448: MINIFICPP-682 - C API: provide functions ...
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239106929 --- Diff: nanofi/include/api/nanofi.h --- @@ -71,62 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The default strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @return 0 (success), -1 (strategy cannot be set - no failure callback added?) + **/ int set_failure_strategy(flow *flow, FailureStrategy strategy); -int set_property(processor *, const char *, const char *); +/** + * Set property for a proces
[GitHub] nifi-minifi-cpp pull request #448: MINIFICPP-682 - C API: provide functions ...
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239105538 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! --- End diff -- "transferred to the caller" of the callback or of this function? ---
[GitHub] nifi-minifi-cpp pull request #448: MINIFICPP-682 - C API: provide functions ...
Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239105384 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, --- End diff -- The problematic case is flow being instantiated properly, but the processor doesn't (the name is wrong for eg.). In this case a valid flow pointer is returned, but the flow doesn't contain any processor. To make it worse: -There is no function in the API to get (at least the number) of processors in the flow -As a valid ptr was returned, the caller could expect everything to be fine, but that's not the case. ---
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710215#comment-16710215 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239106929 --- Diff: nanofi/include/api/nanofi.h --- @@ -71,62 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The default strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @ret
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710205#comment-16710205 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239105538 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! --- End diff -- "transferred to the caller" of the callback or of this function? > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This m
[GitHub] nifi-minifi-cpp pull request #448: MINIFICPP-682 - C API: provide functions ...
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239107442 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The defailt strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @return 0 (success), -1 (strategy cannot be set - no failure callback added?) + **/ int set_failure_strategy(flow *flow, FailureStrategy strategy); -int set_property(processor *, const char *, const char *); +/** + * Set property for a proces
[jira] [Commented] (NIFI-5868) Instrument robust timing information for ListFile
[ https://issues.apache.org/jira/browse/NIFI-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710183#comment-16710183 ] ASF GitHub Bot commented on NIFI-5868: -- Github user bbende commented on the issue: https://github.com/apache/nifi/pull/3202 Will review... > Instrument robust timing information for ListFile > - > > Key: NIFI-5868 > URL: https://issues.apache.org/jira/browse/NIFI-5868 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.9.0 > > > ListFile is used in many different contexts. We often see users with a > specific use case, though, which is to run ListFile on a Primary Node in a > cluster, in order to obtain a file listing of an NFS-mounted share. This > works well in most cases, but whenever problems do arise, it is very > difficult to understand what the problem is. It would be very helpful to have > information such as: > * Is there a problem accessing a specific file on the NFS mount? > * Is there a problem obtaining a listing from the NFS mount? > * Is progress being made at all? > * How long is a listing taking right now? > * How long does a listing typically take? > * Is this problem related to NiFi or to the operating system / > infrastructure? > It would be helpful to track information about each disk access that is > occurring, as well as the overall listing progress and issue warnings if we > see clear problems arise. We can do this by timing how long each disk access > takes, what file was being accessed, and what operating was being performed. > If we capture this data in a rolling window, we can assess the data to > determine if the listing is now taking longer than it did previously and > alert to this fact. Or alert if performing a specific disk operation is > taking a long time. > Gathering this information will likely be fairly heap intensive, so it is > best to make the functionality optional. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710211#comment-16710211 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239106406 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! --- End diff -- Good shout, thanks, I will reword this. > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlas
[GitHub] nifi-minifi-cpp pull request #448: MINIFICPP-682 - C API: provide functions ...
Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239106406 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! --- End diff -- Good shout, thanks, I will reword this. ---
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710217#comment-16710217 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239107442 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The defailt strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @ret
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710218#comment-16710218 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239107724 --- Diff: nanofi/include/api/nanofi.h --- @@ -135,16 +274,62 @@ flow_file_record* create_ff_object(const char *file, const size_t len, const uin flow_file_record* create_ff_object_na(const char *file, const size_t len, const uint64_t size); -void free_flowfile(flow_file_record*); +/** + * Get incoming flow file. To be used in processor logic callbacks. + * @param session current processor session + * @param context current processor context + * @return a flow file record or nullptr in case there is none in the session + **/ +flow_file_record* get_flowfile(processor_session* session, processor_context* context); --- End diff -- So get has been replaced by get_flowfile ? Any reason we can't use this in python? > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710213#comment-16710213 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239106621 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The defailt strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @ret
[GitHub] nifi-minifi-cpp pull request #448: MINIFICPP-682 - C API: provide functions ...
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239106621 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The defailt strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @return 0 (success), -1 (strategy cannot be set - no failure callback added?) + **/ int set_failure_strategy(flow *flow, FailureStrategy strategy); -int set_property(processor *, const char *, const char *); +/** + * Set property for a proces
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710220#comment-16710220 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239108187 --- Diff: nanofi/include/api/nanofi.h --- @@ -135,16 +274,62 @@ flow_file_record* create_ff_object(const char *file, const size_t len, const uin flow_file_record* create_ff_object_na(const char *file, const size_t len, const uint64_t size); -void free_flowfile(flow_file_record*); +/** + * Get incoming flow file. To be used in processor logic callbacks. + * @param session current processor session + * @param context current processor context + * @return a flow file record or nullptr in case there is none in the session + **/ +flow_file_record* get_flowfile(processor_session* session, processor_context* context); + + +/** + * Free flow file + * @param ff flow file + **/ +void free_flowfile(flow_file_record* ff); +/** + * Adds an attribute, fails in case there is already an attribute with the given key. + * @param ff flow file + * @param key name of attribute + * @param value location of value + * @size size size of the data pointed by "value" + * @return 0 in case of success, -1 otherwise (already existed) + **/ uint8_t add_attribute(flow_file_record*, const char *key, void *value, size_t size); -void update_attribute(flow_file_record*, const char *key, void *value, size_t size); +/** + * Updates an attribute (adds if it hasn't existed before) + * @param ff flow file + * @param key name of attribute + * @param value location of value + * @size size size of the data pointed by "value" + **/ +void update_attribute(flow_file_record* ff, const char *key, void *value, size_t size); -uint8_t get_attribute(flow_file_record *ff, attribute *caller_attribute); +/** + * Get the value of an attribute. Value and value size are written to parameter "caller_attribute" + * @param ff flow file + * @param caller_attribute attribute structure to provide name and get value, size + * @return 0 in case of success, -1 otherwise (no such attribute) + **/ +uint8_t get_attribute(const flow_file_record *ff, attribute *caller_attribute); +/** + * Get the quantity of attributes + * @param ff flow file + * @return the number of attributes + **/ int get_attribute_qty(const flow_file_record* ff); --- End diff -- I thought there was a ticket for this ( can't find it so maybe it was never made ?? ) but we shouldn't be using qty or similar within an API. This should be count or quantity. > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #448: MINIFICPP-682 - C API: provide functions ...
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239108187 --- Diff: nanofi/include/api/nanofi.h --- @@ -135,16 +274,62 @@ flow_file_record* create_ff_object(const char *file, const size_t len, const uin flow_file_record* create_ff_object_na(const char *file, const size_t len, const uint64_t size); -void free_flowfile(flow_file_record*); +/** + * Get incoming flow file. To be used in processor logic callbacks. + * @param session current processor session + * @param context current processor context + * @return a flow file record or nullptr in case there is none in the session + **/ +flow_file_record* get_flowfile(processor_session* session, processor_context* context); + + +/** + * Free flow file + * @param ff flow file + **/ +void free_flowfile(flow_file_record* ff); +/** + * Adds an attribute, fails in case there is already an attribute with the given key. + * @param ff flow file + * @param key name of attribute + * @param value location of value + * @size size size of the data pointed by "value" + * @return 0 in case of success, -1 otherwise (already existed) + **/ uint8_t add_attribute(flow_file_record*, const char *key, void *value, size_t size); -void update_attribute(flow_file_record*, const char *key, void *value, size_t size); +/** + * Updates an attribute (adds if it hasn't existed before) + * @param ff flow file + * @param key name of attribute + * @param value location of value + * @size size size of the data pointed by "value" + **/ +void update_attribute(flow_file_record* ff, const char *key, void *value, size_t size); -uint8_t get_attribute(flow_file_record *ff, attribute *caller_attribute); +/** + * Get the value of an attribute. Value and value size are written to parameter "caller_attribute" + * @param ff flow file + * @param caller_attribute attribute structure to provide name and get value, size + * @return 0 in case of success, -1 otherwise (no such attribute) + **/ +uint8_t get_attribute(const flow_file_record *ff, attribute *caller_attribute); +/** + * Get the quantity of attributes + * @param ff flow file + * @return the number of attributes + **/ int get_attribute_qty(const flow_file_record* ff); --- End diff -- I thought there was a ticket for this ( can't find it so maybe it was never made ?? ) but we shouldn't be using qty or similar within an API. This should be count or quantity. ---
[GitHub] nifi-minifi-cpp pull request #448: MINIFICPP-682 - C API: provide functions ...
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239107724 --- Diff: nanofi/include/api/nanofi.h --- @@ -135,16 +274,62 @@ flow_file_record* create_ff_object(const char *file, const size_t len, const uin flow_file_record* create_ff_object_na(const char *file, const size_t len, const uint64_t size); -void free_flowfile(flow_file_record*); +/** + * Get incoming flow file. To be used in processor logic callbacks. + * @param session current processor session + * @param context current processor context + * @return a flow file record or nullptr in case there is none in the session + **/ +flow_file_record* get_flowfile(processor_session* session, processor_context* context); --- End diff -- So get has been replaced by get_flowfile ? Any reason we can't use this in python? ---
[GitHub] nifi-minifi-cpp pull request #448: MINIFICPP-682 - C API: provide functions ...
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239110104
--- Diff: nanofi/include/core/cstructs.h ---
@@ -114,4 +126,6 @@ typedef enum FS {
ROLLBACK
} FailureStrategy;
+typedef void (processor_logic)(processor_session*, processor_context *);
--- End diff --
You have a comment, below, about accessing properties not being supported.
Does that mean that processor_logic will go in tandem with processor
configuration somehow?
---
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[
https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710227#comment-16710227
]
ASF GitHub Bot commented on MINIFICPP-682:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239110104
--- Diff: nanofi/include/core/cstructs.h ---
@@ -114,4 +126,6 @@ typedef enum FS {
ROLLBACK
} FailureStrategy;
+typedef void (processor_logic)(processor_session*, processor_context *);
--- End diff --
You have a comment, below, about accessing properties not being supported.
Does that mean that processor_logic will go in tandem with processor
configuration somehow?
> C API: provide functions to create custom processors
>
>
> Key: MINIFICPP-682
> URL: https://issues.apache.org/jira/browse/MINIFICPP-682
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Major
> Labels: CAPI, nanofi
> Fix For: 0.6.0
>
>
> Extend C API to:
> -Provide functions that can be used used to implement custom processor.
> -Custom processor should be able to read/update both the content and the
> attributes of flowfile, route to "failure" and "success" relationships.
> -API should support adding these custom processors to flows and invoke them
> as standalones, too.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #448: MINIFICPP-682 - C API: provide functions ...
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239109360 --- Diff: nanofi/include/api/nanofi.h --- @@ -165,6 +355,31 @@ uint8_t remove_attribute(flow_file_record*, char *key); int transmit_flowfile(flow_file_record *, nifi_instance *); +/** + * Adds a custom processor for later instantiation + * @param name name of the processor + * @param logic the callback to be invoked when the processor is triggered + * @return 0 on success, -1 otherwise (name already in use for eg.) + **/ +int add_custom_processor(const char * name, processor_logic* logic); --- End diff -- processor_logic doesn't encapsulate the potential of "scheduling" May want to keep that fundamentally so that we allow configuration items to be loaded only once. The Python code left that as a todo ---
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710224#comment-16710224 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239109360 --- Diff: nanofi/include/api/nanofi.h --- @@ -165,6 +355,31 @@ uint8_t remove_attribute(flow_file_record*, char *key); int transmit_flowfile(flow_file_record *, nifi_instance *); +/** + * Adds a custom processor for later instantiation + * @param name name of the processor + * @param logic the callback to be invoked when the processor is triggered + * @return 0 on success, -1 otherwise (name already in use for eg.) + **/ +int add_custom_processor(const char * name, processor_logic* logic); --- End diff -- processor_logic doesn't encapsulate the potential of "scheduling" May want to keep that fundamentally so that we allow configuration items to be loaded only once. The Python code left that as a todo > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #448: MINIFICPP-682 - C API: provide functions ...
Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239111667 --- Diff: nanofi/include/api/nanofi.h --- @@ -165,6 +355,31 @@ uint8_t remove_attribute(flow_file_record*, char *key); int transmit_flowfile(flow_file_record *, nifi_instance *); +/** + * Adds a custom processor for later instantiation + * @param name name of the processor + * @param logic the callback to be invoked when the processor is triggered + * @return 0 on success, -1 otherwise (name already in use for eg.) + **/ +int add_custom_processor(const char * name, processor_logic* logic); --- End diff -- I don't see the simple way of doing this. The part I see complicated is registering static properties in a custom processor, so we should rely on dynamic ones, however those require evaluation in every trigger by their nature. How would you do this? ---
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710232#comment-16710232 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239111667 --- Diff: nanofi/include/api/nanofi.h --- @@ -165,6 +355,31 @@ uint8_t remove_attribute(flow_file_record*, char *key); int transmit_flowfile(flow_file_record *, nifi_instance *); +/** + * Adds a custom processor for later instantiation + * @param name name of the processor + * @param logic the callback to be invoked when the processor is triggered + * @return 0 on success, -1 otherwise (name already in use for eg.) + **/ +int add_custom_processor(const char * name, processor_logic* logic); --- End diff -- I don't see the simple way of doing this. The part I see complicated is registering static properties in a custom processor, so we should rely on dynamic ones, however those require evaluation in every trigger by their nature. How would you do this? > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #448: MINIFICPP-682 - C API: provide functions ...
Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239113983 --- Diff: nanofi/include/api/nanofi.h --- @@ -135,16 +274,62 @@ flow_file_record* create_ff_object(const char *file, const size_t len, const uin flow_file_record* create_ff_object_na(const char *file, const size_t len, const uint64_t size); -void free_flowfile(flow_file_record*); +/** + * Get incoming flow file. To be used in processor logic callbacks. + * @param session current processor session + * @param context current processor context + * @return a flow file record or nullptr in case there is none in the session + **/ +flow_file_record* get_flowfile(processor_session* session, processor_context* context); --- End diff -- I think we can, will give it a try ---
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710234#comment-16710234 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239113983 --- Diff: nanofi/include/api/nanofi.h --- @@ -135,16 +274,62 @@ flow_file_record* create_ff_object(const char *file, const size_t len, const uin flow_file_record* create_ff_object_na(const char *file, const size_t len, const uint64_t size); -void free_flowfile(flow_file_record*); +/** + * Get incoming flow file. To be used in processor logic callbacks. + * @param session current processor session + * @param context current processor context + * @return a flow file record or nullptr in case there is none in the session + **/ +flow_file_record* get_flowfile(processor_session* session, processor_context* context); --- End diff -- I think we can, will give it a try > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #448: MINIFICPP-682 - C API: provide functions ...
Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239115752 --- Diff: nanofi/include/api/nanofi.h --- @@ -71,62 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The default strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @return 0 (success), -1 (strategy cannot be set - no failure callback added?) + **/ int set_failure_strategy(flow *flow, FailureStrategy strategy); -int set_property(processor *, const char *, const char *); +/** + * Set property for a proce
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710240#comment-16710240 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239115752 --- Diff: nanofi/include/api/nanofi.h --- @@ -71,62 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The default strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @re
[GitHub] nifi-minifi-cpp pull request #448: MINIFICPP-682 - C API: provide functions ...
Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239116973 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The defailt strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @return 0 (success), -1 (strategy cannot be set - no failure callback added?) + **/ int set_failure_strategy(flow *flow, FailureStrategy strategy); -int set_property(processor *, const char *, const char *); +/** + * Set property for a proce
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710241#comment-16710241 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239116973 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The defailt strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @re
[GitHub] nifi-minifi-cpp pull request #455: MINIFICPP-689 - Make minifi::Exception co...
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239117109
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
--- End diff --
Just saw this when I was typing the one below, I'll run the build before
merging obviously, but does this not generate an elision warning?
---
[GitHub] nifi-minifi-cpp pull request #455: MINIFICPP-689 - Make minifi::Exception co...
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239117254
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
}
+
// Destructor
- virtual ~Exception() throw () {
+ virtual ~Exception() noexcept {
--- End diff --
I didn't notice that throw was here. Odd choice. The compiler usually
generates an implicit noexcept unless told otherwise, so I'm tempted to do a
little more testing of this across compilers. I'll do this before merge.
---
[jira] [Commented] (MINIFICPP-689) Make minifi::Exception constructible with string param
[
https://issues.apache.org/jira/browse/MINIFICPP-689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710244#comment-16710244
]
ASF GitHub Bot commented on MINIFICPP-689:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239117254
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
}
+
// Destructor
- virtual ~Exception() throw () {
+ virtual ~Exception() noexcept {
--- End diff --
I didn't notice that throw was here. Odd choice. The compiler usually
generates an implicit noexcept unless told otherwise, so I'm tempted to do a
little more testing of this across compilers. I'll do this before merge.
> Make minifi::Exception constructible with string param
> --
>
> Key: MINIFICPP-689
> URL: https://issues.apache.org/jira/browse/MINIFICPP-689
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Minor
> Fix For: 0.6.0
>
>
> Exception is currently only constructible using const char * argument, but
> that's copied into a string. Using string parameter would make it more
> developer-friendly and save some copy constructions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (NIFI-5872) Add compression option on JsonRecordSetWriter
Pierre Villard created NIFI-5872: Summary: Add compression option on JsonRecordSetWriter Key: NIFI-5872 URL: https://issues.apache.org/jira/browse/NIFI-5872 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Pierre Villard Add a compression option on the JsonRecordSetWriter controller service as we have on the AvroRecordSetWriter controller service. This can be useful in case of very IO intensive workflows. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-689) Make minifi::Exception constructible with string param
[
https://issues.apache.org/jira/browse/MINIFICPP-689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710243#comment-16710243
]
ASF GitHub Bot commented on MINIFICPP-689:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239117109
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
--- End diff --
Just saw this when I was typing the one below, I'll run the build before
merging obviously, but does this not generate an elision warning?
> Make minifi::Exception constructible with string param
> --
>
> Key: MINIFICPP-689
> URL: https://issues.apache.org/jira/browse/MINIFICPP-689
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Minor
> Fix For: 0.6.0
>
>
> Exception is currently only constructible using const char * argument, but
> that's copied into a string. Using string parameter would make it more
> developer-friendly and save some copy constructions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710258#comment-16710258 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239123478 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, --- End diff -- Ok, removed deprecation. > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #448: MINIFICPP-682 - C API: provide functions ...
Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239123478 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, --- End diff -- Ok, removed deprecation. ---
[GitHub] nifi-minifi-cpp pull request #448: MINIFICPP-682 - C API: provide functions ...
Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239125843 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The defailt strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @return 0 (success), -1 (strategy cannot be set - no failure callback added?) + **/ int set_failure_strategy(flow *flow, FailureStrategy strategy); -int set_property(processor *, const char *, const char *); +/** + * Set property for a proce
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710265#comment-16710265 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239125843 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The defailt strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @re
[GitHub] nifi-minifi-cpp pull request #455: MINIFICPP-689 - Make minifi::Exception co...
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239140994
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
}
+
// Destructor
- virtual ~Exception() throw () {
+ virtual ~Exception() noexcept {
--- End diff --
throw() == noexcept, however throw() is deprecated since C++11.
---
[jira] [Commented] (MINIFICPP-689) Make minifi::Exception constructible with string param
[
https://issues.apache.org/jira/browse/MINIFICPP-689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710296#comment-16710296
]
ASF GitHub Bot commented on MINIFICPP-689:
--
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239140994
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
}
+
// Destructor
- virtual ~Exception() throw () {
+ virtual ~Exception() noexcept {
--- End diff --
throw() == noexcept, however throw() is deprecated since C++11.
> Make minifi::Exception constructible with string param
> --
>
> Key: MINIFICPP-689
> URL: https://issues.apache.org/jira/browse/MINIFICPP-689
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Minor
> Fix For: 0.6.0
>
>
> Exception is currently only constructible using const char * argument, but
> that's copied into a string. Using string parameter would make it more
> developer-friendly and save some copy constructions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #455: MINIFICPP-689 - Make minifi::Exception co...
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239140370
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
--- End diff --
As far as I remember that warning is only applied for return values, where
moving could prevent copy-elision, but it's not the case.
---
[jira] [Commented] (MINIFICPP-689) Make minifi::Exception constructible with string param
[
https://issues.apache.org/jira/browse/MINIFICPP-689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710295#comment-16710295
]
ASF GitHub Bot commented on MINIFICPP-689:
--
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239140370
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
--- End diff --
As far as I remember that warning is only applied for return values, where
moving could prevent copy-elision, but it's not the case.
> Make minifi::Exception constructible with string param
> --
>
> Key: MINIFICPP-689
> URL: https://issues.apache.org/jira/browse/MINIFICPP-689
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Minor
> Fix For: 0.6.0
>
>
> Exception is currently only constructible using const char * argument, but
> that's copied into a string. Using string parameter would make it more
> developer-friendly and save some copy constructions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-689) Make minifi::Exception constructible with string param
[
https://issues.apache.org/jira/browse/MINIFICPP-689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710307#comment-16710307
]
ASF GitHub Bot commented on MINIFICPP-689:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239144107
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
}
+
// Destructor
- virtual ~Exception() throw () {
+ virtual ~Exception() noexcept {
--- End diff --
I'm not sure how that applies to my comment...but I think the end result is
that I will still run basic tests across compilers. Are you suggesting we
needn't run tests for verification and validation?
> Make minifi::Exception constructible with string param
> --
>
> Key: MINIFICPP-689
> URL: https://issues.apache.org/jira/browse/MINIFICPP-689
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Minor
> Fix For: 0.6.0
>
>
> Exception is currently only constructible using const char * argument, but
> that's copied into a string. Using string parameter would make it more
> developer-friendly and save some copy constructions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #455: MINIFICPP-689 - Make minifi::Exception co...
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239144107
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
}
+
// Destructor
- virtual ~Exception() throw () {
+ virtual ~Exception() noexcept {
--- End diff --
I'm not sure how that applies to my comment...but I think the end result is
that I will still run basic tests across compilers. Are you suggesting we
needn't run tests for verification and validation?
---
[GitHub] nifi-minifi-cpp pull request #455: MINIFICPP-689 - Make minifi::Exception co...
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239144665
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
--- End diff --
Ah yah. I don't remember either, but that sounds right If I don't see the
warning when I run the build then all is well that ends well. Thanks.
---
[jira] [Commented] (MINIFICPP-689) Make minifi::Exception constructible with string param
[
https://issues.apache.org/jira/browse/MINIFICPP-689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710314#comment-16710314
]
ASF GitHub Bot commented on MINIFICPP-689:
--
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239145213
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
}
+
// Destructor
- virtual ~Exception() throw () {
+ virtual ~Exception() noexcept {
--- End diff --
No, I wouldn't skip them. Just meant that the change should be identical,
so if it worked, it should still work _in theory_, but we are engineers, so
let's see the results. :)
> Make minifi::Exception constructible with string param
> --
>
> Key: MINIFICPP-689
> URL: https://issues.apache.org/jira/browse/MINIFICPP-689
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Minor
> Fix For: 0.6.0
>
>
> Exception is currently only constructible using const char * argument, but
> that's copied into a string. Using string parameter would make it more
> developer-friendly and save some copy constructions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-689) Make minifi::Exception constructible with string param
[
https://issues.apache.org/jira/browse/MINIFICPP-689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710312#comment-16710312
]
ASF GitHub Bot commented on MINIFICPP-689:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239144665
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
--- End diff --
Ah yah. I don't remember either, but that sounds right If I don't see the
warning when I run the build then all is well that ends well. Thanks.
> Make minifi::Exception constructible with string param
> --
>
> Key: MINIFICPP-689
> URL: https://issues.apache.org/jira/browse/MINIFICPP-689
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Minor
> Fix For: 0.6.0
>
>
> Exception is currently only constructible using const char * argument, but
> that's copied into a string. Using string parameter would make it more
> developer-friendly and save some copy constructions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #455: MINIFICPP-689 - Make minifi::Exception co...
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/455#discussion_r239145213
--- Diff: libminifi/include/Exception.h ---
@@ -60,16 +60,17 @@ class Exception : public std::exception {
public:
// Constructor
/*!
- * Create a new flow record
+ * Create a new exception
*/
- Exception(ExceptionType type, const char *errorMsg)
+ Exception(ExceptionType type, std::string errorMsg)
: _type(type),
-_errorMsg(errorMsg) {
+_errorMsg(std::move(errorMsg)) {
}
+
// Destructor
- virtual ~Exception() throw () {
+ virtual ~Exception() noexcept {
--- End diff --
No, I wouldn't skip them. Just meant that the change should be identical,
so if it worked, it should still work _in theory_, but we are engineers, so
let's see the results. :)
---
[GitHub] nifi pull request #3203: NIFI-5871 ignore UUID attribute when copying flow f...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3203#discussion_r239160879 --- Diff: nifi-mock/src/main/java/org/apache/nifi/util/MockProcessSession.java --- @@ -496,7 +496,10 @@ public MockFlowFile putAllAttributes(FlowFile flowFile, final Map attrCopy = new HashMap<>(); +attrCopy.putAll(attrs); --- End diff -- Only change I'd make here is putting that into the constructor. Other than that LGTM. ---
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710366#comment-16710366 ] Peter Wicks commented on NIFI-4621: --- Hi [~kislayom], Your PR does not look correct to me. Did you check-in all of the commit's, maybe something got lost? Right now I don't see most of the changes I expected to see (EL changes you mentioned before are missing, etc...) > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Peter Wicks >Priority: Critical > Fix For: 1.9.0 > > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #3105: NIFI-5621: Added Cassandra connection provider serv...
Github user zenfenan commented on a diff in the pull request: https://github.com/apache/nifi/pull/3105#discussion_r239163988 --- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-services-nar/src/main/resources/META-INF/NOTICE --- @@ -0,0 +1,327 @@ +nifi-cassandra-services-nar --- End diff -- Fixed all. Please take a look. ---
[jira] [Commented] (NIFI-5621) Create a Connection Pooling service implementation to be used in Cassandra processors
[ https://issues.apache.org/jira/browse/NIFI-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710381#comment-16710381 ] ASF GitHub Bot commented on NIFI-5621: -- Github user zenfenan commented on a diff in the pull request: https://github.com/apache/nifi/pull/3105#discussion_r239164057 --- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-services-nar/src/main/resources/META-INF/NOTICE --- @@ -0,0 +1,327 @@ +nifi-cassandra-services-nar +Copyright 2016 The Apache Software Foundation + +This product includes software developed at +The Apache Software Foundation (http://www.apache.org/). + +** +Apache Software License v2 +** + +The following binary components are provided under the Apache Software License v2 + + (ASLv2) DataStax Java Driver for Apache Cassandra - Core + The following NOTICE information applies: +DataStax Java Driver for Apache Cassandra - Core +Copyright (C) 2012-2017 DataStax Inc. + + (ASLv2) Apache Avro + The following NOTICE information applies: +Apache Avro +Copyright 2009-2017 The Apache Software Foundation + + (ASLv2) Jackson JSON processor --- End diff -- Understood. > Create a Connection Pooling service implementation to be used in Cassandra > processors > - > > Key: NIFI-5621 > URL: https://issues.apache.org/jira/browse/NIFI-5621 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Sivaprasanna Sethuraman >Assignee: Sivaprasanna Sethuraman >Priority: Major > > Like how the Relational Database processors leverage 'DBCPConnectionPool' > controller service, there should be one that could be used by the processors > from Cassandra bundle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
