[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711572#comment-16711572 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/448 > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711571#comment-16711571 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/448 Also added https://issues.apache.org/jira/browse/MINIFICPP-695 since examples don't appear to work in testing. Merging anyway so we can fix that as a follow on > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711545#comment-16711545 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239481840 --- Diff: nanofi/include/api/nanofi.h --- @@ -165,6 +355,31 @@ uint8_t remove_attribute(flow_file_record*, char *key); int transmit_flowfile(flow_file_record *, nifi_instance *); +/** + * Adds a custom processor for later instantiation + * @param name name of the processor + * @param logic the callback to be invoked when the processor is triggered + * @return 0 on success, -1 otherwise (name already in use for eg.) + **/ +int add_custom_processor(const char * name, processor_logic* logic); --- End diff -- Typically the session can define a mapping of properties and can call a "scheduling" piece before every iteration. In some cases, this could be at the outset of execution of a custom processor. Effectively before their actions are invoked, the processor should be configured. We can make that stipulation a priori unless properties are dynamic, in which case we've sort of tabled that for the interim. MINIFICPP-694 is a good follow on to that. > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711532#comment-16711532 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239479166 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The defailt strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @return 0 (success),
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711523#comment-16711523 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239477701 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, --- End diff -- Could we not change it to return a null pointer? > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711521#comment-16711521 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239477181 --- Diff: nanofi/include/api/nanofi.h --- @@ -71,62 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The default strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @return 0 (success),
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710265#comment-16710265 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239125843 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The defailt strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @return 0
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710258#comment-16710258 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239123478 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, --- End diff -- Ok, removed deprecation. > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710241#comment-16710241 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239116973 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The defailt strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @return 0
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710240#comment-16710240 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239115752 --- Diff: nanofi/include/api/nanofi.h --- @@ -71,62 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The default strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @return 0
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710234#comment-16710234 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239113983 --- Diff: nanofi/include/api/nanofi.h --- @@ -135,16 +274,62 @@ flow_file_record* create_ff_object(const char *file, const size_t len, const uin flow_file_record* create_ff_object_na(const char *file, const size_t len, const uint64_t size); -void free_flowfile(flow_file_record*); +/** + * Get incoming flow file. To be used in processor logic callbacks. + * @param session current processor session + * @param context current processor context + * @return a flow file record or nullptr in case there is none in the session + **/ +flow_file_record* get_flowfile(processor_session* session, processor_context* context); --- End diff -- I think we can, will give it a try > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710232#comment-16710232 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239111667 --- Diff: nanofi/include/api/nanofi.h --- @@ -165,6 +355,31 @@ uint8_t remove_attribute(flow_file_record*, char *key); int transmit_flowfile(flow_file_record *, nifi_instance *); +/** + * Adds a custom processor for later instantiation + * @param name name of the processor + * @param logic the callback to be invoked when the processor is triggered + * @return 0 on success, -1 otherwise (name already in use for eg.) + **/ +int add_custom_processor(const char * name, processor_logic* logic); --- End diff -- I don't see the simple way of doing this. The part I see complicated is registering static properties in a custom processor, so we should rely on dynamic ones, however those require evaluation in every trigger by their nature. How would you do this? > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710224#comment-16710224 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239109360 --- Diff: nanofi/include/api/nanofi.h --- @@ -165,6 +355,31 @@ uint8_t remove_attribute(flow_file_record*, char *key); int transmit_flowfile(flow_file_record *, nifi_instance *); +/** + * Adds a custom processor for later instantiation + * @param name name of the processor + * @param logic the callback to be invoked when the processor is triggered + * @return 0 on success, -1 otherwise (name already in use for eg.) + **/ +int add_custom_processor(const char * name, processor_logic* logic); --- End diff -- processor_logic doesn't encapsulate the potential of "scheduling" May want to keep that fundamentally so that we allow configuration items to be loaded only once. The Python code left that as a todo > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[
https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710227#comment-16710227
]
ASF GitHub Bot commented on MINIFICPP-682:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239110104
--- Diff: nanofi/include/core/cstructs.h ---
@@ -114,4 +126,6 @@ typedef enum FS {
ROLLBACK
} FailureStrategy;
+typedef void (processor_logic)(processor_session*, processor_context *);
--- End diff --
You have a comment, below, about accessing properties not being supported.
Does that mean that processor_logic will go in tandem with processor
configuration somehow?
> C API: provide functions to create custom processors
>
>
> Key: MINIFICPP-682
> URL: https://issues.apache.org/jira/browse/MINIFICPP-682
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Major
> Labels: CAPI, nanofi
> Fix For: 0.6.0
>
>
> Extend C API to:
> -Provide functions that can be used used to implement custom processor.
> -Custom processor should be able to read/update both the content and the
> attributes of flowfile, route to "failure" and "success" relationships.
> -API should support adding these custom processors to flows and invoke them
> as standalones, too.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710220#comment-16710220 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239108187 --- Diff: nanofi/include/api/nanofi.h --- @@ -135,16 +274,62 @@ flow_file_record* create_ff_object(const char *file, const size_t len, const uin flow_file_record* create_ff_object_na(const char *file, const size_t len, const uint64_t size); -void free_flowfile(flow_file_record*); +/** + * Get incoming flow file. To be used in processor logic callbacks. + * @param session current processor session + * @param context current processor context + * @return a flow file record or nullptr in case there is none in the session + **/ +flow_file_record* get_flowfile(processor_session* session, processor_context* context); + + +/** + * Free flow file + * @param ff flow file + **/ +void free_flowfile(flow_file_record* ff); +/** + * Adds an attribute, fails in case there is already an attribute with the given key. + * @param ff flow file + * @param key name of attribute + * @param value location of value + * @size size size of the data pointed by "value" + * @return 0 in case of success, -1 otherwise (already existed) + **/ uint8_t add_attribute(flow_file_record*, const char *key, void *value, size_t size); -void update_attribute(flow_file_record*, const char *key, void *value, size_t size); +/** + * Updates an attribute (adds if it hasn't existed before) + * @param ff flow file + * @param key name of attribute + * @param value location of value + * @size size size of the data pointed by "value" + **/ +void update_attribute(flow_file_record* ff, const char *key, void *value, size_t size); -uint8_t get_attribute(flow_file_record *ff, attribute *caller_attribute); +/** + * Get the value of an attribute. Value and value size are written to parameter "caller_attribute" + * @param ff flow file + * @param caller_attribute attribute structure to provide name and get value, size + * @return 0 in case of success, -1 otherwise (no such attribute) + **/ +uint8_t get_attribute(const flow_file_record *ff, attribute *caller_attribute); +/** + * Get the quantity of attributes + * @param ff flow file + * @return the number of attributes + **/ int get_attribute_qty(const flow_file_record* ff); --- End diff -- I thought there was a ticket for this ( can't find it so maybe it was never made ?? ) but we shouldn't be using qty or similar within an API. This should be count or quantity. > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710213#comment-16710213 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239106621 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The defailt strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @return 0 (success),
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710217#comment-16710217 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239107442 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The defailt strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @return 0 (success),
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710218#comment-16710218 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239107724 --- Diff: nanofi/include/api/nanofi.h --- @@ -135,16 +274,62 @@ flow_file_record* create_ff_object(const char *file, const size_t len, const uin flow_file_record* create_ff_object_na(const char *file, const size_t len, const uint64_t size); -void free_flowfile(flow_file_record*); +/** + * Get incoming flow file. To be used in processor logic callbacks. + * @param session current processor session + * @param context current processor context + * @return a flow file record or nullptr in case there is none in the session + **/ +flow_file_record* get_flowfile(processor_session* session, processor_context* context); --- End diff -- So get has been replaced by get_flowfile ? Any reason we can't use this in python? > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710211#comment-16710211 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239106406 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! --- End diff -- Good shout, thanks, I will reword this. > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710205#comment-16710205 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239105538 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! --- End diff -- "transferred to the caller" of the callback or of this function? > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710215#comment-16710215 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239106929 --- Diff: nanofi/include/api/nanofi.h --- @@ -71,62 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The default strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @return 0 (success),
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710204#comment-16710204 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239105384 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, --- End diff -- The problematic case is flow being instantiated properly, but the processor doesn't (the name is wrong for eg.). In this case a valid flow pointer is returned, but the flow doesn't contain any processor. To make it worse: -There is no function in the API to get (at least the number) of processors in the flow -As a valid ptr was returned, the caller could expect everything to be fine, but that's not the case. > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710200#comment-16710200 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239105138 --- Diff: nanofi/include/api/nanofi.h --- @@ -71,62 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); --- End diff -- This was starved off due to other work, but we will eventually need directory specific accessor functions for languages , I would imagine. This could be a candidate for that. > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710189#comment-16710189 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239103770 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, --- End diff -- I'm unclear about deprecation. Does this mean that it's deprecated because you have no way of indicating the processor could not be instantiated? Wouldn't that arrive at the case where flow is null? Flow in create_new_flow could be null by virtue of a malloc error, so why deprecate a function that results in the same behavior and fewer function calls? > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710170#comment-16710170 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/448 taking a look > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710191#comment-16710191 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r239104013 --- Diff: nanofi/include/api/nanofi.h --- @@ -68,60 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, --- End diff -- This is precisely something we want to keep by design. Let's talk about this before merging this. > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707431#comment-16707431 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r238330333 --- Diff: nanofi/include/api/nanofi.h --- @@ -71,62 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); -standalone_processor *create_processor(const char *); +/** + * Create a standalone instance of the given processor. + * Standalone instances can be invoked without having an instance/flow that contains them. + * @param name the name of the processor to instanciate + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + **/ +standalone_processor *create_processor(const char * name); -void free_standalone_processor(standalone_processor*); +/** + * Free a standalone processor + * @param processor the processor to be freed + */ +void free_standalone_processor(standalone_processor* processor); /** -* Register your callback to received flow files that the flow failed to process -* The flow file ownership is transferred to the caller! -* The first callback should be registered before the flow is used. Can be changed later during runtime. -*/ + * Register your callback to received flow files that the flow failed to process + * The flow file ownership is transferred to the caller! + * The first callback should be registered before the flow is used. Can be changed later during runtime. + * @param flow flow the callback belongs to + * @param onerror_callback callback to execute in case of failure + * @return 0 in case of success, -1 otherwise (flow is already in use) + **/ int add_failure_callback(flow *flow, void (*onerror_callback)(flow_file_record*)); - /** -* Set failure strategy. Please use the enum defined in cstructs.h -* Return values: 0 (success), -1 (strategy cannot be set - no failure callback added?) -* Can be changed runtime. -* The default strategy is AS IS. -*/ + * Set failure strategy. Please use the enum defined in cstructs.h + * Can be changed runtime. + * The default strategy is AS IS. + * @param flow the flow to set strategy for + * @param strategy the strategy to be set + * @return 0
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707428#comment-16707428 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r238329859 --- Diff: nanofi/include/api/nanofi.h --- @@ -71,62 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); - -flow *create_new_flow(nifi_instance *); -flow *create_flow(nifi_instance *, const char *); +/** + * Creates new flow and adds the first processor in case a valid name is provided + * @deprecated as there is no proper indication of processor adding errors, + * usage of "create_new_flow" and "add_processor is recommended instead + * @param instance the instance new flow will belong to + * @param first_processor name of the first processor to be instanciated + * @attention in case first processor is empty or doesn't name any existing processor, an empty flow is returned. + * @return a pointer to the created flow + **/ +DEPRECATED flow *create_flow(nifi_instance * instance, const char * first_processor); -flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); +/** + * Add a getfile processor to "parent" flow. + * Creates new flow in instance in case "parent" is nullptr + * @deprecated as getfile processor can be added using "add_processor" function, + * properties can be set using "set_property". + * @param instance the instance the flow belongs to + * @param parent the flow to be extended with a new getfile processor + * @param c configuration of the new processor + * @return parent in case it wasn't null, otherwise a pointer to a new flow + */ +DEPRECATED flow *create_getfile(nifi_instance *instance, flow *parent, GetFileConfig *c); -processor *add_processor(flow *, const char *); +/** + * Extend a flow with a new processor + * @param flow the flow to be extended with the new processor + * @param name name of the new processor + * @return pointer to the new processor or nullptr in case it cannot be instantiated (wrong name?) + */ +processor *add_processor(flow * flow, const char * name); processor *add_python_processor(flow *, void (*ontrigger_callback)(processor_session *session)); --- End diff -- Didn't document as I'm not sure this is the right place for this. This is only used by Python libraries, so I don't think this should be public and documented. > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707423#comment-16707423 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r238329139 --- Diff: nanofi/include/api/nanofi.h --- @@ -71,62 +94,173 @@ typedef int c2_start_callback(char *); void enable_async_c2(nifi_instance *, C2_Server *, c2_stop_callback *, c2_start_callback *, c2_update_callback *); +/** + * Creates a new, empty flow + * @param instance the instance new flow will belong to + * @return a pointer to the created flow + **/ +flow *create_new_flow(nifi_instance * instance); -uint8_t run_processor(const processor *processor); --- End diff -- I removed this as it has no implementation, so pretty sure noone ever used. > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16706941#comment-16706941 ] ASF GitHub Bot commented on MINIFICPP-682: -- Github user arpadboda commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/448 > I just skimmed the latest commit. Really good stuff. This will go a long way toward helping consumers of this API. Very excited! > > Is this ready for review? If so I'll try to take a look this weekend sometime. Thanks! Not yet, plan to finish today. > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[
https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16702041#comment-16702041
]
ASF GitHub Bot commented on MINIFICPP-682:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r237139770
--- Diff: nanofi/src/api/nanofi.cpp ---
@@ -223,13 +226,15 @@ void free_flowfile(flow_file_record *ff) {
return;
}
auto content_repo_ptr =
static_cast*>(ff->crp);
- if (content_repo_ptr->get()) {
+ if (content_repo_ptr->get() && (ff->keepContent == 0)) {
--- End diff --
Yeah that seems like it could cause issues when freeing the flow file, but
if they were to do all of this, this means that they've either threaded the
reads of content or are calling free in the middle of accessing that content? I
think it's reasonable for each read to "keep the content" because it's not the
owner. A better way might be to use the reference counting in ResourceClaim but
we don't presently have that since we make new claim objects. I think what
you've done is not unreasonable.
> C API: provide functions to create custom processors
>
>
> Key: MINIFICPP-682
> URL: https://issues.apache.org/jira/browse/MINIFICPP-682
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Major
> Labels: CAPI, nanofi
> Fix For: 0.6.0
>
>
> Extend C API to:
> -Provide functions that can be used used to implement custom processor.
> -Custom processor should be able to read/update both the content and the
> attributes of flowfile, route to "failure" and "success" relationships.
> -API should support adding these custom processors to flows and invoke them
> as standalones, too.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[
https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16698590#comment-16698590
]
ASF GitHub Bot commented on MINIFICPP-682:
--
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r236150567
--- Diff: nanofi/include/cxx/CallbackProcessor.h ---
@@ -65,24 +67,19 @@ class CallbackProcessor : public core::Processor {
public:
- void setCallback(void *obj,std::function
ontrigger_callback) {
+ void setCallback(void *obj,std::function ontrigger_callback) {
objref_ = obj;
callback_ = ontrigger_callback;
}
// OnTrigger method, implemented by NiFi CallbackProcessor
- virtual void onTrigger(core::ProcessContext *context,
core::ProcessSession *session);
+ virtual void onTrigger(core::ProcessContext *context,
core::ProcessSession *session) override;
// Initialize, over write by NiFi CallbackProcessor
- virtual void initialize() {
-std::set relationships;
-core::Relationship Success("success", "description");
-relationships.insert(Success);
-setSupportedRelationships(relationships);
- }
+ virtual void initialize() override;
protected:
void *objref_;
- std::function callback_;
+ std::function callback_;
--- End diff --
Context is required in the callback to:
-Access content
-Access properties (not yet supported, but will be)
> C API: provide functions to create custom processors
>
>
> Key: MINIFICPP-682
> URL: https://issues.apache.org/jira/browse/MINIFICPP-682
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Major
> Labels: CAPI, nanofi
> Fix For: 0.6.0
>
>
> Extend C API to:
> -Provide functions that can be used used to implement custom processor.
> -Custom processor should be able to read/update both the content and the
> attributes of flowfile, route to "failure" and "success" relationships.
> -API should support adding these custom processors to flows and invoke them
> as standalones, too.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[
https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16698587#comment-16698587
]
ASF GitHub Bot commented on MINIFICPP-682:
--
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/448#discussion_r236149200
--- Diff: nanofi/src/api/nanofi.cpp ---
@@ -223,13 +226,15 @@ void free_flowfile(flow_file_record *ff) {
return;
}
auto content_repo_ptr =
static_cast*>(ff->crp);
- if (content_repo_ptr->get()) {
+ if (content_repo_ptr->get() && (ff->keepContent == 0)) {
--- End diff --
@phrocker : I don't link this, but I have no better idea yet. We provide a
flow_file_record to access content and attributes in the processor logic
callback. This means that the content cannot be deleted in case the flowfile is
transfered via a relationship.
> C API: provide functions to create custom processors
>
>
> Key: MINIFICPP-682
> URL: https://issues.apache.org/jira/browse/MINIFICPP-682
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Major
> Labels: CAPI, nanofi
> Fix For: 0.6.0
>
>
> Extend C API to:
> -Provide functions that can be used used to implement custom processor.
> -Custom processor should be able to read/update both the content and the
> attributes of flowfile, route to "failure" and "success" relationships.
> -API should support adding these custom processors to flows and invoke them
> as standalones, too.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16698583#comment-16698583 ] ASF GitHub Bot commented on MINIFICPP-682: -- GitHub user arpadboda opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/448 MINIFICPP-682 - C API: provide functions to create custom processors Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/arpadboda/nifi-minifi-cpp MINIFICPP-682 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/448.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #448 commit ea330109394b57d3fc76a7c414195b8c8808d0a5 Author: Arpad Boda Date: 2018-11-26T07:50:28Z MINIFICPP-682 - C API: provide functions to create custom processors > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-682) C API: provide functions to create custom processors
[ https://issues.apache.org/jira/browse/MINIFICPP-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691609#comment-16691609 ] Mr TheSegfault commented on MINIFICPP-682: -- [~aboda] I think this is a great start. Zero copy is something to think about ( maybe not necessarily implement yet ) – this may come in the form of using system calls ( something like mmap ) or just transferring pointers where it makes sense ( if it makes sense ). > C API: provide functions to create custom processors > > > Key: MINIFICPP-682 > URL: https://issues.apache.org/jira/browse/MINIFICPP-682 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > Extend C API to: > -Provide functions that can be used used to implement custom processor. > -Custom processor should be able to read/update both the content and the > attributes of flowfile, route to "failure" and "success" relationships. > -API should support adding these custom processors to flows and invoke them > as standalones, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
