[jira] [Commented] (NIFI-5744) Put exception message to attribute while ExecuteSQL fail
[
https://issues.apache.org/jira/browse/NIFI-5744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689754#comment-16689754
]
ASF GitHub Bot commented on NIFI-5744:
--
Github user yjhyjhyjh0 commented on the issue:
https://github.com/apache/nifi/pull/3107
Thanks for the suggestion.
Rebase against master, solve conflicts and squash into single commit.
> Put exception message to attribute while ExecuteSQL fail
>
>
> Key: NIFI-5744
> URL: https://issues.apache.org/jira/browse/NIFI-5744
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.7.1
>Reporter: Deon Huang
>Assignee: Deon Huang
>Priority: Minor
>
> In some scenario, it would be great if we could have different behavior based
> on exception.
> Better error tracking afterwards in attribute format instead of tracking in
> log.
> For example, if it’s connection refused exception due to wrong url.
> We won’t want to retry and error message attribute would be helpful to keep
> track of.
> While it’s other scenario that database temporary unavailable, we should
> retry it based on should retry exception.
> Should be a quick fix at AbstractExecuteSQL before transfer flowfile to
> failure relationship
> {code:java}
> session.transfer(fileToProcess, REL_FAILURE);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5820) NiFi built with Java 1.8 needs to run on Java 11
[ https://issues.apache.org/jira/browse/NIFI-5820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689614#comment-16689614 ] ASF GitHub Bot commented on NIFI-5820: -- Github user jtstorck commented on the issue: https://github.com/apache/nifi/pull/3174 Thanks for taking a look at the PR, @joewitt. That warning is due to accessing, via reflection, the `pid` method on the Process API, which was added in Java 9. The code that does this was added by NIFI-5175, to allow NiFI built on Java 1.8 to run on Java 9. There's a comment in the code detailing why the use of reflection is necessary. Please see https://github.com/apache/nifi/blob/master/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/util/OSUtils.java#L111. The warning is expected, and when we have a minimum requirement of Java 11, we can refactor OSUtils, or probably remove the class entirely, since the Process API (as of Java 9) provides a platform independent way to get a PID. I don't think we'll need to have the methods in OSUtils for getting the PID based on which platform on which NiFi is running. > NiFi built with Java 1.8 needs to run on Java 11 > > > Key: NIFI-5820 > URL: https://issues.apache.org/jira/browse/NIFI-5820 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Jeff Storck >Assignee: Jeff Storck >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-645) Move from new to malloc in CAPI to facilitate eventual change from C++ to C
[
https://issues.apache.org/jira/browse/MINIFICPP-645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689490#comment-16689490
]
ASF GitHub Bot commented on MINIFICPP-645:
--
Github user phrocker commented on the issue:
https://github.com/apache/nifi-minifi-cpp/pull/439
> There is no delete, it was wrong before, this PR just fixes:
>
> ```
> void free_flowfile(flow_file_record *ff) {
> if (ff == nullptr) {
> return;
> }
> auto content_repo_ptr =
static_cast*>(ff->crp);
> if (content_repo_ptr->get()) {
> std::shared_ptr claim =
std::make_shared(ff->contentLocation, *content_repo_ptr);
> (*content_repo_ptr)->remove(claim);
> }
> if (ff->ffp == nullptr) {
> auto map = static_cast(ff->attributes);
> delete map;
> }
> free(ff->contentLocation);
> free(ff);
> ```
> The last line is the one that frees.
Ah sorry, I was referencing the fact that over the course of PRs we've gone
back and forth a little between malloc/new. There is a free_flow(flow *) that
still uses delete. Happy to see a different PR if you prefer to do that, but it
all falls under the guise of this ticket IMO. Would you prefer I merge this and
then keep the ticket open as a blocker for the free? No real preference on my
part.
> Move from new to malloc in CAPI to facilitate eventual change from C++ to C
> ---
>
> Key: MINIFICPP-645
> URL: https://issues.apache.org/jira/browse/MINIFICPP-645
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Mr TheSegfault
>Assignee: Arpad Boda
>Priority: Blocker
> Labels: CAPI, nanofi
> Fix For: 0.6.0
>
>
> As gradually move to C we should move out of libminifi and remove the linter.
> Nothing that is returned via the API that is not an opaque pointer should use
> new, and conversely nothing that is passed in as a non-opaque pointer should
> be deleted versus freed
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5820) NiFi built with Java 1.8 needs to run on Java 11
[ https://issues.apache.org/jira/browse/NIFI-5820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689402#comment-16689402 ] ASF GitHub Bot commented on NIFI-5820: -- Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/3174 it built with java 8. switched to 11. nifi starts up and appears to be working great. I did notice this on startup nifi.sh: JAVA_HOME not set; results may vary Java home: NiFi home: /Users/joe/development/nifi.git/nifi-assembly/target/nifi-1.9.0-SNAPSHOT-bin/nifi-1.9.0-SNAPSHOT Bootstrap Config File: /../development/nifi.git/nifi-assembly/target/nifi-1.9.0-SNAPSHOT-bin/nifi-1.9.0-SNAPSHOT/conf/bootstrap.conf WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.apache.nifi.bootstrap.util.OSUtils (file:/../development/nifi.git/nifi-assembly/target/nifi-1.9.0-SNAPSHOT-bin/nifi-1.9.0-SNAPSHOT/lib/bootstrap/nifi-bootstrap-1.9.0-SNAPSHOT.jar) to method java.lang.ProcessImpl.pid() WARNING: Please consider reporting this to the maintainers of org.apache.nifi.bootstrap.util.OSUtils WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release > NiFi built with Java 1.8 needs to run on Java 11 > > > Key: NIFI-5820 > URL: https://issues.apache.org/jira/browse/NIFI-5820 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Jeff Storck >Assignee: Jeff Storck >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5662) AvroTypeUtil Decimal support using Fixed Error
[
https://issues.apache.org/jira/browse/NIFI-5662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689387#comment-16689387
]
ASF GitHub Bot commented on NIFI-5662:
--
GitHub user gideonkorir opened a pull request:
https://github.com/apache/nifi/pull/3175
Support for generic fixed when using decimal logical type
Fix for [Avro decimal
conversion](https://jira.apache.org/jira/browse/NIFI-5662)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gideonkorir/nifi nifi-5662
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/3175.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3175
commit b52a3a240fcfbbec288deb37a0ef43acd79e42d2
Author: gkkorir
Date: 2018-11-16T12:49:24Z
Support for generic fixed when using decimal logical type
> AvroTypeUtil Decimal support using Fixed Error
> --
>
> Key: NIFI-5662
> URL: https://issues.apache.org/jira/browse/NIFI-5662
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 1.7.1
> Environment: RHEL 7.5
> JDK 1.8.182
>Reporter: Gideon Korir
>Priority: Major
>
> When the decimal is specified as fixed in the Avro schema, AvroTypeUtils
> converts the decimal into a ByteBuffer instead of a GenericFixed.
> The code:
> {code:java}
> return new Conversions.DecimalConversion().toBytes(decimal, fieldSchema,
> logicalType)
> {code}
> Should be:
> {code:java}
> return fieldSchema.getType() == Type.BYTES
> ? new Conversions.DecimalConversion().toBytes(decimal, fieldSchema,
> logicalType)
> : new Conversions.DecimalConversion().toFixed(decimal, fieldSchema,
> logicalType);
> {code}
> The former causes the AvroRecordSetWriter to fail with the error:
> _org.apache.avro.file.DataFileWriter$AppendWriteException:
> java.lang.ClassCastException: java.nio.HeapByteBuffer cannot be cast to
> org.apache.avro.generic.GenericFixed_
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-676) Cleanup and fix serializable interface implementation
[
https://issues.apache.org/jira/browse/MINIFICPP-676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689313#comment-16689313
]
ASF GitHub Bot commented on MINIFICPP-676:
--
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/440#discussion_r234176325
--- Diff: libminifi/include/io/Serializable.h ---
@@ -22,11 +22,36 @@
#include
#include "EndianCheck.h"
#include "DataStream.h"
+
+namespace {
+ template Cleanup and fix serializable interface implementation
> -
>
> Key: MINIFICPP-676
> URL: https://issues.apache.org/jira/browse/MINIFICPP-676
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Minor
>
> Serializable interface contains a couple of issues:
> Type-unsafe template functions
> Code duplication
> Needless functions
> The goal of this ticket is to make the interface cleaner and simpler.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-645) Move from new to malloc in CAPI to facilitate eventual change from C++ to C
[
https://issues.apache.org/jira/browse/MINIFICPP-645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689301#comment-16689301
]
ASF GitHub Bot commented on MINIFICPP-645:
--
Github user arpadboda commented on the issue:
https://github.com/apache/nifi-minifi-cpp/pull/439
There is no delete, it was wrong before:
```
void free_flowfile(flow_file_record *ff) {
if (ff == nullptr) {
return;
}
auto content_repo_ptr =
static_cast*>(ff->crp);
if (content_repo_ptr->get()) {
std::shared_ptr claim =
std::make_shared(ff->contentLocation, *content_repo_ptr);
(*content_repo_ptr)->remove(claim);
}
if (ff->ffp == nullptr) {
auto map = static_cast(ff->attributes);
delete map;
}
free(ff->contentLocation);
free(ff);
```
The last line is the one that frees.
> Move from new to malloc in CAPI to facilitate eventual change from C++ to C
> ---
>
> Key: MINIFICPP-645
> URL: https://issues.apache.org/jira/browse/MINIFICPP-645
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Mr TheSegfault
>Assignee: Arpad Boda
>Priority: Blocker
> Labels: CAPI, nanofi
> Fix For: 0.6.0
>
>
> As gradually move to C we should move out of libminifi and remove the linter.
> Nothing that is returned via the API that is not an opaque pointer should use
> new
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5823) DBCPConnectionPool minIdleConnections property name has a typo
[
https://issues.apache.org/jira/browse/NIFI-5823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688917#comment-16688917
]
ASF GitHub Bot commented on NIFI-5823:
--
Github user ijokarumawak commented on the issue:
https://github.com/apache/nifi/pull/3172
LGTM +1, merged to master. Good catch before releasing :)
> DBCPConnectionPool minIdleConnections property name has a typo
> --
>
> Key: NIFI-5823
> URL: https://issues.apache.org/jira/browse/NIFI-5823
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.9.0
>Reporter: Colin Dean
>Priority: Major
>
> {{mim}} -> {{min}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5823) DBCPConnectionPool minIdleConnections property name has a typo
[
https://issues.apache.org/jira/browse/NIFI-5823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688916#comment-16688916
]
ASF GitHub Bot commented on NIFI-5823:
--
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/3172
> DBCPConnectionPool minIdleConnections property name has a typo
> --
>
> Key: NIFI-5823
> URL: https://issues.apache.org/jira/browse/NIFI-5823
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.9.0
>Reporter: Colin Dean
>Priority: Major
>
> {{mim}} -> {{min}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5820) NiFi built with Java 1.8 needs to run on Java 11
[ https://issues.apache.org/jira/browse/NIFI-5820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688802#comment-16688802 ] ASF GitHub Bot commented on NIFI-5820: -- GitHub user jtstorck opened a pull request: https://github.com/apache/nifi/pull/3174 [WIP] NIFI-5820 NiFi built on Java 1.8 can run on Java 9/10/11 Updated RunNiFi.java to add libs need to run on Java 11 when it is the detected runtime java version and grant access to the necessary module when running on Java 9 or 10 Added dependencies/includes/excludes to nifi-assembly configurations for enabling NiFi to run on Java 11 Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jtstorck/nifi NIFI-5820 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3174.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3174 commit 4859444b6b503a86953d0a6b06be07f081357552 Author: Jeff Storck Date: 2018-11-15T23:38:02Z NIFI-5820 NiFi built on Java 1.8 can run on Java 9/10/11 Updated RunNiFi.java to add libs need to run on Java 11 when it is the detected runtime java version and grant access to the necessary module when running on Java 9 or 10 Added dependencies/includes/excludes to nifi-assembly configurations for enabling NiFi to run on Java 11 > NiFi built with Java 1.8 needs to run on Java 11 > > > Key: NIFI-5820 > URL: https://issues.apache.org/jira/browse/NIFI-5820 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Jeff Storck >Assignee: Jeff Storck >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-679) Improve Const correctness in core and ID
[
https://issues.apache.org/jira/browse/MINIFICPP-679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688763#comment-16688763
]
ASF GitHub Bot commented on MINIFICPP-679:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/442#discussion_r234040428
--- Diff: libminifi/include/core/Core.h ---
@@ -142,52 +141,30 @@ class CoreComponent {
* Constructor that sets the name and uuid.
*/
- explicit CoreComponent(const std::string name, utils::Identifier uuid)
+ explicit CoreComponent(const std::string , utils::Identifier uuid)
: name_(name) {
if (uuid == nullptr) {
// Generate the global UUID for the flow record
id_generator_->generate(uuid_);
} else {
uuid_ = uuid;
}
-
uuidStr_ = uuid_.to_string();
}
- explicit CoreComponent(const std::string name)
+ explicit CoreComponent(const std::string )
: name_(name) {
-
// Generate the global UUID for the flow record
id_generator_->generate(uuid_);
-
uuidStr_ = uuid_.to_string();
}
- /*
- #ifdef WIN32
- #else
-
- explicit CoreComponent(const std::string name, Identifier uuid =
nullptr)
- : name_(name) {
- if (nullptr == uuid)
- // Generate the global UUID for the flow record
- id_generator_->generate(uuid_);
- else
- uuid_copy(uuid_, uuid);
-
- char uuidStr[37] = { 0 };
- uuid_unparse_lower(uuid_, uuidStr);
- uuidStr_ = uuidStr;
- }
- #endif
- */
+
+ explicit CoreComponent(const CoreComponent ) = default;
/**
* Move Constructor.
*/
- explicit CoreComponent(const CoreComponent &)
- : name_(std::move(other.name_)) {
-uuid_ = other.uuid_;
-//uuid_copy(uuid_, other.uuid_);
- }
+
--- End diff --
should be safe since we added a non trial move con to ident, but this will
require some testing on all platforms. again, still running windows build. Will
probably report back tomorrow.
> Improve Const correctness in core and ID
>
>
> Key: MINIFICPP-679
> URL: https://issues.apache.org/jira/browse/MINIFICPP-679
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Mr TheSegfault
>Assignee: Mr TheSegfault
>Priority: Major
>
> Improve Const correctness in core and ID as a way to kick off MINIFICPP-678
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-679) Improve Const correctness in core and ID
[ https://issues.apache.org/jira/browse/MINIFICPP-679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688715#comment-16688715 ] ASF GitHub Bot commented on MINIFICPP-679: -- GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/442 MINIFICPP-679: Make minor changes for const correctness improvements Running a windows build now. If I see errors there I'll re-submit, but ran general tests and build across a variety of platforms. Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFICPP-679 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/442.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #442 commit 12fe6ccdf163bc585425df28e6bbc7f5cfb6347f Author: Marc Parisi Date: 2018-11-15T22:10:42Z MINIFICPP-679: Make minor changes for const correctness improvements > Improve Const correctness in core and ID > > > Key: MINIFICPP-679 > URL: https://issues.apache.org/jira/browse/MINIFICPP-679 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > > Improve Const correctness in core and ID as a way to kick off MINIFICPP-678 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-676) Cleanup and fix serializable interface implementation
[ https://issues.apache.org/jira/browse/MINIFICPP-676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688689#comment-16688689 ] ASF GitHub Bot commented on MINIFICPP-676: -- Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/440 hmmm, I should have checked appveyor because I would have saved some time building on windows...but as I suspected we have issues there too. > Cleanup and fix serializable interface implementation > - > > Key: MINIFICPP-676 > URL: https://issues.apache.org/jira/browse/MINIFICPP-676 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Minor > > Serializable interface contains a couple of issues: > Type-unsafe template functions > Code duplication > Needless functions > The goal of this ticket is to make the interface cleaner and simpler. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-645) Move from new to malloc in CAPI to facilitate eventual change from C++ to C
[
https://issues.apache.org/jira/browse/MINIFICPP-645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688673#comment-16688673
]
ASF GitHub Bot commented on MINIFICPP-645:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/439#discussion_r234012390
--- Diff: nanofi/src/api/nanofi.cpp ---
@@ -205,7 +205,7 @@ flow_file_record* create_ff_object(const char *file,
const size_t len, const uin
}
flow_file_record* create_ff_object_na(const char *file, const size_t len,
const uint64_t size) {
- flow_file_record *new_ff = new flow_file_record;
+ flow_file_record *new_ff = (flow_file_record*)
malloc(sizeof(flow_file_record));
--- End diff --
Oops, sorry, must change the delete to free due to avoid heap corruption.
> Move from new to malloc in CAPI to facilitate eventual change from C++ to C
> ---
>
> Key: MINIFICPP-645
> URL: https://issues.apache.org/jira/browse/MINIFICPP-645
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Mr TheSegfault
>Assignee: Arpad Boda
>Priority: Blocker
> Labels: CAPI, nanofi
> Fix For: 0.6.0
>
>
> As gradually move to C we should move out of libminifi and remove the linter.
> Nothing that is returned via the API that is not an opaque pointer should use
> new
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-667) Create Structural definitions for moving away from C++ classes
[ https://issues.apache.org/jira/browse/MINIFICPP-667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688668#comment-16688668 ] ASF GitHub Bot commented on MINIFICPP-667: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/436#discussion_r234011794 --- Diff: nanofi/include/core/utlist.h --- @@ -0,0 +1,1073 @@ +/* --- End diff -- As per the title this is to work out a plan so that it's out in the open on Apache. I won't be merging this. > Create Structural definitions for moving away from C++ classes > -- > > Key: MINIFICPP-667 > URL: https://issues.apache.org/jira/browse/MINIFICPP-667 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: nanofi >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > Fix For: 0.6.0 > > > Create Structural definitions for moving away from C++ classes -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-676) Cleanup and fix serializable interface implementation
[
https://issues.apache.org/jira/browse/MINIFICPP-676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688658#comment-16688658
]
ASF GitHub Bot commented on MINIFICPP-676:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/440#discussion_r234008835
--- Diff: libminifi/include/io/Serializable.h ---
@@ -173,8 +147,22 @@ class Serializable {
**/
int readUTF(std::string , DataStream *stream, bool widen = false);
- protected:
+ /**
+ * reads 2-8 bytes from the stream
+ * @param value reference in which will set the result
+ * @param stream stream from which we will read
+ * @return resulting read size
+ **/
+ template 1) &&
+ std::is_integral::value &&
+ !std::is_signed::value
+ ,Integral>::type* = nullptr>
+ int read(Integral , DataStream *stream, bool is_little_endian =
EndiannessCheck::IS_LITTLE) {
--- End diff --
My IDEs does not make this clear the function to use. In some ways java
made this easier with their streams "writeInt, writeShort, writeUnsignedShort"
and we can do the same with write(short) write(int), write(long), but my one
IDE correctly provides this while others don't. I agree that generally
reducing "Needless functions" is a good thing, but here it served a purpose.
Happy to further discuss how we can benefit that purpose.
> Cleanup and fix serializable interface implementation
> -
>
> Key: MINIFICPP-676
> URL: https://issues.apache.org/jira/browse/MINIFICPP-676
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Minor
>
> Serializable interface contains a couple of issues:
> Type-unsafe template functions
> Code duplication
> Needless functions
> The goal of this ticket is to make the interface cleaner and simpler.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-676) Cleanup and fix serializable interface implementation
[
https://issues.apache.org/jira/browse/MINIFICPP-676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688666#comment-16688666
]
ASF GitHub Bot commented on MINIFICPP-676:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/440#discussion_r234011459
--- Diff: extensions/http-curl/client/HTTPCallback.h ---
@@ -78,7 +78,7 @@ class HttpStreamingCallback : public ByteInputCallBack {
}
- virtual int64_t process(uint8_t *vector, size_t size) {
+ virtual int64_t process(const uint8_t *vector, size_t size) {
--- End diff --
in general when i see something like const uint8_t *ptr, then I wonder what
the true intent may be there. You have a pointer that can be modified. Should
also make that pointer const too, but as with all comments I'm not sure that's
the best approach here.
> Cleanup and fix serializable interface implementation
> -
>
> Key: MINIFICPP-676
> URL: https://issues.apache.org/jira/browse/MINIFICPP-676
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Minor
>
> Serializable interface contains a couple of issues:
> Type-unsafe template functions
> Code duplication
> Needless functions
> The goal of this ticket is to make the interface cleaner and simpler.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-676) Cleanup and fix serializable interface implementation
[
https://issues.apache.org/jira/browse/MINIFICPP-676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688660#comment-16688660
]
ASF GitHub Bot commented on MINIFICPP-676:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/440#discussion_r234009476
--- Diff: libminifi/src/io/FileStream.cpp ---
@@ -82,16 +82,16 @@ void FileStream::seek(uint64_t offset) {
file_stream_->seekp(offset_);
}
-int FileStream::writeData(std::vector , int buflen) {
+int FileStream::writeData(const std::vector , int buflen) {
if (static_cast(buf.capacity()) < buflen) {
return -1;
}
- return writeData(reinterpret_cast([0]), buflen);
--- End diff --
fails to compile on some of my archs. perhaps run make docker ?
> Cleanup and fix serializable interface implementation
> -
>
> Key: MINIFICPP-676
> URL: https://issues.apache.org/jira/browse/MINIFICPP-676
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Minor
>
> Serializable interface contains a couple of issues:
> Type-unsafe template functions
> Code duplication
> Needless functions
> The goal of this ticket is to make the interface cleaner and simpler.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-676) Cleanup and fix serializable interface implementation
[
https://issues.apache.org/jira/browse/MINIFICPP-676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688659#comment-16688659
]
ASF GitHub Bot commented on MINIFICPP-676:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/440#discussion_r234009236
--- Diff: libminifi/include/io/tls/TLSSocket.h ---
@@ -173,19 +173,19 @@ class TLSSocket : public Socket {
* @param buflen buffer to write
*
*/
- int writeData(std::vector , int buflen);
+ int writeData(const std::vector , int buflen);
--- End diff --
if we chose ( and we eventually will in further tickets ) to take this
memory and do a memmove ( in a different stream -- can discuss later ) we can't
do that with the changes. This was intentional and not an oversight.
> Cleanup and fix serializable interface implementation
> -
>
> Key: MINIFICPP-676
> URL: https://issues.apache.org/jira/browse/MINIFICPP-676
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Minor
>
> Serializable interface contains a couple of issues:
> Type-unsafe template functions
> Code duplication
> Needless functions
> The goal of this ticket is to make the interface cleaner and simpler.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-676) Cleanup and fix serializable interface implementation
[
https://issues.apache.org/jira/browse/MINIFICPP-676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688655#comment-16688655
]
ASF GitHub Bot commented on MINIFICPP-676:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/440#discussion_r234008240
--- Diff: libminifi/include/io/Serializable.h ---
@@ -22,11 +22,36 @@
#include
#include "EndianCheck.h"
#include "DataStream.h"
+
+namespace {
+ template Cleanup and fix serializable interface implementation
> -
>
> Key: MINIFICPP-676
> URL: https://issues.apache.org/jira/browse/MINIFICPP-676
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Minor
>
> Serializable interface contains a couple of issues:
> Type-unsafe template functions
> Code duplication
> Needless functions
> The goal of this ticket is to make the interface cleaner and simpler.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-676) Cleanup and fix serializable interface implementation
[
https://issues.apache.org/jira/browse/MINIFICPP-676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688651#comment-16688651
]
ASF GitHub Bot commented on MINIFICPP-676:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/440#discussion_r234007116
--- Diff: libminifi/include/io/CRCStream.h ---
@@ -194,15 +194,12 @@ int CRCStream::readData(uint8_t *buf, int buflen) {
}
template
-int CRCStream::writeData(std::vector , int buflen) {
-
- if ((int)buf.capacity() < buflen)
--- End diff --
This existed for a reason. Did you test this in flight ? This causes a
failure. We have non const because we reserve the right to adjust buffers.
That's the intent behind making the signatures the way they are.
> Cleanup and fix serializable interface implementation
> -
>
> Key: MINIFICPP-676
> URL: https://issues.apache.org/jira/browse/MINIFICPP-676
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Minor
>
> Serializable interface contains a couple of issues:
> Type-unsafe template functions
> Code duplication
> Needless functions
> The goal of this ticket is to make the interface cleaner and simpler.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-676) Cleanup and fix serializable interface implementation
[ https://issues.apache.org/jira/browse/MINIFICPP-676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688637#comment-16688637 ] ASF GitHub Bot commented on MINIFICPP-676: -- Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/440 @arpadboda I've run some flows and seen a lot of failures as well. > Cleanup and fix serializable interface implementation > - > > Key: MINIFICPP-676 > URL: https://issues.apache.org/jira/browse/MINIFICPP-676 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Minor > > Serializable interface contains a couple of issues: > Type-unsafe template functions > Code duplication > Needless functions > The goal of this ticket is to make the interface cleaner and simpler. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-676) Cleanup and fix serializable interface implementation
[
https://issues.apache.org/jira/browse/MINIFICPP-676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688649#comment-16688649
]
ASF GitHub Bot commented on MINIFICPP-676:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/440#discussion_r234006270
--- Diff: extensions/http-curl/client/HTTPStream.h ---
@@ -100,14 +100,14 @@ class HttpStream : public io::BaseStream {
* @param buflen buffer to write
*
*/
- virtual int writeData(std::vector , int buflen);
+ virtual int writeData(const std::vector , int buflen);
--- End diff --
Same. C and C++ developers see const and they know what that means. They
see the lack of const and know their data may change. If it's non const that
doesn't mean it should be changed without discussion. I think these should be
non const as we reserve the right to take ownership in the internal API. I
would agree for external API.
> Cleanup and fix serializable interface implementation
> -
>
> Key: MINIFICPP-676
> URL: https://issues.apache.org/jira/browse/MINIFICPP-676
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Minor
>
> Serializable interface contains a couple of issues:
> Type-unsafe template functions
> Code duplication
> Needless functions
> The goal of this ticket is to make the interface cleaner and simpler.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-676) Cleanup and fix serializable interface implementation
[
https://issues.apache.org/jira/browse/MINIFICPP-676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688646#comment-16688646
]
ASF GitHub Bot commented on MINIFICPP-676:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/440#discussion_r234005818
--- Diff: extensions/http-curl/client/HTTPStream.cpp ---
@@ -53,16 +53,16 @@ void HttpStream::seek(uint64_t offset) {
throw std::exception();
}
-int HttpStream::writeData(std::vector , int buflen) {
+int HttpStream::writeData(const std::vector , int buflen) {
if ((int)buf.capacity() < buflen) {
return -1;
}
- return writeData(reinterpret_cast([0]), buflen);
--- End diff --
this fails on some compilers. I spent a long time and saw failures on older
versions ( not sure if specifically this one ). Did this cause a specific bug?
These are non const for a reason and that's to tell the caller that their data
may change. You're making an assumption that it shouldn't be const.
> Cleanup and fix serializable interface implementation
> -
>
> Key: MINIFICPP-676
> URL: https://issues.apache.org/jira/browse/MINIFICPP-676
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Minor
>
> Serializable interface contains a couple of issues:
> Type-unsafe template functions
> Code duplication
> Needless functions
> The goal of this ticket is to make the interface cleaner and simpler.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5604) Allow GenerateTableFetch to send empty flow files when no rows would be fetched
[ https://issues.apache.org/jira/browse/NIFI-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688616#comment-16688616 ] ASF GitHub Bot commented on NIFI-5604: -- Github user patricker commented on the issue: https://github.com/apache/nifi/pull/3075 +1 LGTM. I tested this out with several different scenarios. After our chat on Slack, I squashed this down to a single commit under your name to make things cleaner. > Allow GenerateTableFetch to send empty flow files when no rows would be > fetched > --- > > Key: NIFI-5604 > URL: https://issues.apache.org/jira/browse/NIFI-5604 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > > Currently, GenerateTableFetch will not output a flow file if there are no > rows to be fetched. However, it may be desired (especially with incoming flow > files) that a flow file be sent out even if GTF does not generate any SQL. > This capability, along with the fragment attributes from NIFI-5601, would > allow the user to handle this downstream. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5744) Put exception message to attribute while ExecuteSQL fail
[
https://issues.apache.org/jira/browse/NIFI-5744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688620#comment-16688620
]
ASF GitHub Bot commented on NIFI-5744:
--
Github user patricker commented on the issue:
https://github.com/apache/nifi/pull/3107
Hey, @yjhyjhyjh0, can you rebase against master and push a new squashed
commit? I think this is ready to merge if we resolve this new code conflict.
> Put exception message to attribute while ExecuteSQL fail
>
>
> Key: NIFI-5744
> URL: https://issues.apache.org/jira/browse/NIFI-5744
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.7.1
>Reporter: Deon Huang
>Assignee: Deon Huang
>Priority: Minor
>
> In some scenario, it would be great if we could have different behavior based
> on exception.
> Better error tracking afterwards in attribute format instead of tracking in
> log.
> For example, if it’s connection refused exception due to wrong url.
> We won’t want to retry and error message attribute would be helpful to keep
> track of.
> While it’s other scenario that database temporary unavailable, we should
> retry it based on should retry exception.
> Should be a quick fix at AbstractExecuteSQL before transfer flowfile to
> failure relationship
> {code:java}
> session.transfer(fileToProcess, REL_FAILURE);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5604) Allow GenerateTableFetch to send empty flow files when no rows would be fetched
[ https://issues.apache.org/jira/browse/NIFI-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688617#comment-16688617 ] ASF GitHub Bot commented on NIFI-5604: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/3075 > Allow GenerateTableFetch to send empty flow files when no rows would be > fetched > --- > > Key: NIFI-5604 > URL: https://issues.apache.org/jira/browse/NIFI-5604 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > > Currently, GenerateTableFetch will not output a flow file if there are no > rows to be fetched. However, it may be desired (especially with incoming flow > files) that a flow file be sent out even if GTF does not generate any SQL. > This capability, along with the fragment attributes from NIFI-5601, would > allow the user to handle this downstream. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5780) Add pre and post statements to ExecuteSQL and ExecuteSQLRecord
[ https://issues.apache.org/jira/browse/NIFI-5780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688596#comment-16688596 ] ASF GitHub Bot commented on NIFI-5780: -- Github user patricker commented on the issue: https://github.com/apache/nifi/pull/3156 +1 Looks good. Built and tested. I ran into one issue that I might submit a seperate enhancement for (allowing `;` in the SQL string if they are surrounded by `'` or `"`). > Add pre and post statements to ExecuteSQL and ExecuteSQLRecord > -- > > Key: NIFI-5780 > URL: https://issues.apache.org/jira/browse/NIFI-5780 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.8.0 >Reporter: Deon Huang >Assignee: Deon Huang >Priority: Minor > > Sometimes we might need to set up session relate configuration before or > after query. > For example: > Pre query can be used for session relate setting like our use case Teradata > Query Banding. > Same feature (pre query and post query) is added to SelectHiveQL in > https://issues.apache.org/jira/browse/NIFI-5044 > Planning to add this feature to ExecuteSQL and ExecuteSQLRecord processors. > If pre or post statement fail, will not produce resultset flowfile. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5780) Add pre and post statements to ExecuteSQL and ExecuteSQLRecord
[ https://issues.apache.org/jira/browse/NIFI-5780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688593#comment-16688593 ] ASF GitHub Bot commented on NIFI-5780: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/3156 > Add pre and post statements to ExecuteSQL and ExecuteSQLRecord > -- > > Key: NIFI-5780 > URL: https://issues.apache.org/jira/browse/NIFI-5780 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.8.0 >Reporter: Deon Huang >Assignee: Deon Huang >Priority: Minor > > Sometimes we might need to set up session relate configuration before or > after query. > For example: > Pre query can be used for session relate setting like our use case Teradata > Query Banding. > Same feature (pre query and post query) is added to SelectHiveQL in > https://issues.apache.org/jira/browse/NIFI-5044 > Planning to add this feature to ExecuteSQL and ExecuteSQLRecord processors. > If pre or post statement fail, will not produce resultset flowfile. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5824) Add test to FlowController to ensure that NIFI-5822 is properly resolved
[ https://issues.apache.org/jira/browse/NIFI-5824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688572#comment-16688572 ] ASF GitHub Bot commented on NIFI-5824: -- Github user bbende commented on the issue: https://github.com/apache/nifi/pull/3173 Code looks good and tests pass, will merge > Add test to FlowController to ensure that NIFI-5822 is properly resolved > > > Key: NIFI-5824 > URL: https://issues.apache.org/jira/browse/NIFI-5824 > Project: Apache NiFi > Issue Type: Test > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.9.0 > > > While the fix for NIFI-5822 has been merged and addresses the issue, it would > be helpful to have a unit test to verify it, so that we can ensure that we > never get into this situation again. It's a slight change that could easily > be undone in the future and it's not obvious until specific processors > attempt to run. We should have a unit test to verify that the Process > Scheduler is able to properly access the ControllerServiceProvider -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5824) Add test to FlowController to ensure that NIFI-5822 is properly resolved
[ https://issues.apache.org/jira/browse/NIFI-5824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688543#comment-16688543 ] ASF GitHub Bot commented on NIFI-5824: -- GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/3173 NIFI-5824: Added unit test to FlowController to ensure that the Proce… …ssScheduler that it creates is properly initialized. Also updated the properties file used by TestFlowController to use a VolatileContentRepository instead of FileSystemRepository, and fixed EventDrivenWorkerQueue to return if calls to poll() are interrupted (via Thread.interrupt) - making these minor fixes resulted in the unit test TestFlowController running in 2 seconds instead of 30+ seconds on my machine Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-5824 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3173.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3173 commit ecdcbba8d31af75b3406f336e5272ec3e9a2be21 Author: Mark Payne Date: 2018-11-15T19:26:36Z NIFI-5824: Added unit test to FlowController to ensure that the ProcessScheduler that it creates is properly initialized. Also updated the properties file used by TestFlowController to use a VolatileContentRepository instead of FileSystemRepository, and fixed EventDrivenWorkerQueue to return if calls to poll() are interrupted (via Thread.interrupt) - making these minor fixes resulted in the unit test TestFlowController running in 2 seconds instead of 30+ seconds on my machine > Add test to FlowController to ensure that NIFI-5822 is properly resolved > > > Key: NIFI-5824 > URL: https://issues.apache.org/jira/browse/NIFI-5824 > Project: Apache NiFi > Issue Type: Test > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.9.0 > > > While the fix for NIFI-5822 has been merged and addresses the issue, it would > be helpful to have a unit test to verify it, so that we can ensure that we > never get into this situation again. It's a slight change that could easily > be undone in the future and it's not obvious until specific processors > attempt to run. We should have a unit test to verify that the Process > Scheduler is able to properly access the ControllerServiceProvider -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5823) DBCPConnectionPool minIdleConnections property name has a typo
[
https://issues.apache.org/jira/browse/NIFI-5823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688504#comment-16688504
]
ASF GitHub Bot commented on NIFI-5823:
--
GitHub user colindean opened a pull request:
https://github.com/apache/nifi/pull/3172
NIFI-5823: Fixes typo in min idle connections property name
So many people missed it :-(
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [X] Does your PR title start with NIFI- where is the JIRA number
you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [X] Has your PR been rebased against the latest commit within the target
branch (typically master)?
- [X] Is your initial contribution a single, squashed commit?
### For code changes:
- [X] Have you ensured that the full suite of tests is executed via mvn
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] ~If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?~
- [ ] ~If applicable, have you updated the LICENSE file, including the main
LICENSE file under nifi-assembly?~
- [ ] ~If applicable, have you updated the NOTICE file, including the main
NOTICE file found under nifi-assembly?~
- [ ] ~If adding new Properties, have you added .displayName in addition to
.name (programmatic access) for each of the new properties?~
### For documentation related changes:
- [ ] ~Have you ensured that format looks appropriate for the output in
which it is rendered?~
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/colindean/nifi patch-1
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/3172.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3172
commit 09794d1506541808541e56285ddb5cfa3dbf5fec
Author: Colin Dean
Date: 2018-11-15T18:46:41Z
NIFI-5823: Fixes typo in min idle connections property name
So many people missed it :-(
> DBCPConnectionPool minIdleConnections property name has a typo
> --
>
> Key: NIFI-5823
> URL: https://issues.apache.org/jira/browse/NIFI-5823
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.9.0
>Reporter: Colin Dean
>Priority: Major
>
> {{mim}} -> {{min}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5780) Add pre and post statements to ExecuteSQL and ExecuteSQLRecord
[ https://issues.apache.org/jira/browse/NIFI-5780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688498#comment-16688498 ] ASF GitHub Bot commented on NIFI-5780: -- Github user patricker commented on the issue: https://github.com/apache/nifi/pull/3156 Sorry for the delay, been having issues with a bug that just got fixed in NIFI-5822. Will wrap up testing. > Add pre and post statements to ExecuteSQL and ExecuteSQLRecord > -- > > Key: NIFI-5780 > URL: https://issues.apache.org/jira/browse/NIFI-5780 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.8.0 >Reporter: Deon Huang >Assignee: Deon Huang >Priority: Minor > > Sometimes we might need to set up session relate configuration before or > after query. > For example: > Pre query can be used for session relate setting like our use case Teradata > Query Banding. > Same feature (pre query and post query) is added to SelectHiveQL in > https://issues.apache.org/jira/browse/NIFI-5044 > Planning to add this feature to ExecuteSQL and ExecuteSQLRecord processors. > If pre or post statement fail, will not produce resultset flowfile. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5822) ControllerServices not available during @OnScheduled
[ https://issues.apache.org/jira/browse/NIFI-5822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688479#comment-16688479 ] ASF GitHub Bot commented on NIFI-5822: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/3171 > ControllerServices not available during @OnScheduled > > > Key: NIFI-5822 > URL: https://issues.apache.org/jira/browse/NIFI-5822 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.9.0 >Reporter: Peter Wicks >Assignee: Mark Payne >Priority: Blocker > Fix For: 1.9.0 > > > If a processor tries to load a ControllerService during `@OnScheduled`, then > `context.getControllerService` contains a null reference to > `controllerServiceProvider`. I traced this back to > `StandardProcessScheduler.startProcessor`, and the reference to > `controllerServiceProvider` is also `null` here. `ExecuteSQL` acquires it's > connection during `@OnScheduled`. > But once we reach `onTrigger`, everything is fine and you can retrieve a > controller service without issue -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5822) ControllerServices not available during @OnScheduled
[ https://issues.apache.org/jira/browse/NIFI-5822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688477#comment-16688477 ] ASF GitHub Bot commented on NIFI-5822: -- Github user bbende commented on the issue: https://github.com/apache/nifi/pull/3171 +1 code looks good, verified functionality, will merge > ControllerServices not available during @OnScheduled > > > Key: NIFI-5822 > URL: https://issues.apache.org/jira/browse/NIFI-5822 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.9.0 >Reporter: Peter Wicks >Assignee: Mark Payne >Priority: Blocker > Fix For: 1.9.0 > > > If a processor tries to load a ControllerService during `@OnScheduled`, then > `context.getControllerService` contains a null reference to > `controllerServiceProvider`. I traced this back to > `StandardProcessScheduler.startProcessor`, and the reference to > `controllerServiceProvider` is also `null` here. `ExecuteSQL` acquires it's > connection during `@OnScheduled`. > But once we reach `onTrigger`, everything is fine and you can retrieve a > controller service without issue -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-558) Move PayloadSerializer in preparation for Coap
[
https://issues.apache.org/jira/browse/MINIFICPP-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688452#comment-16688452
]
ASF GitHub Bot commented on MINIFICPP-558:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/437#discussion_r233951286
--- Diff: extensions/coap/controllerservice/CoapConnector.cpp ---
@@ -0,0 +1,189 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "CoapConnector.h"
+#include
+#include
+#include
+#include
+#include
+#include "core/Property.h"
+#include "CoapConnector.h"
+#include "io/validation.h"
+#include "properties/Configure.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace controllers {
+
+static core::Property RemoteServer;
+static core::Property Port;
+static core::Property MaxQueueSize;
+
+core::Property
CoapConnectorService::RemoteServer(core::PropertyBuilder::createProperty("Remote
Server")->withDescription("Remote CoAP server")->isRequired(true)->build());
+core::Property
CoapConnectorService::Port(core::PropertyBuilder::createProperty("Remote
Port")->withDescription("Remote CoAP server port")->isRequired(true)->build());
+core::Property
CoapConnectorService::MaxQueueSize(core::PropertyBuilder::createProperty("Max
Queue Size")->withDescription("Max queue size for received data
")->isRequired(true)->build());
+
+void CoapConnectorService::initialize() {
+ if (initialized_)
+return;
+
+ callback_pointers ptrs;
+ ptrs.data_received = receiveMessage;
+ ptrs.received_error = receiveError;
+ init_coap_api(this, );
+
+ std::lock_guard lock(initialization_mutex_);
+
+ ControllerService::initialize();
+
+ initializeProperties();
+
+ initialized_ = true;
+}
+
+void CoapConnectorService::onEnable() {
+ std::string port_str;
+ if (getProperty(RemoteServer.getName(), host_) && !host_.empty() &&
getProperty(Port.getName(), port_str) && !port_str.empty()) {
+core::Property::StringToInt(port_str, port_);
+ } else {
+// this is the case where we aren't being used in the context of a
single controller service.
+if (configuration_->get("nifi.c2.agent.coap.host", host_) &&
configuration_->get("nifi.c2.agent.coap.port", port_str)) {
+ core::Property::StringToInt(port_str, port_);
+}
+
+ }
+}
+
+CoapConnectorService::CoAPMessage
CoapConnectorService::sendPayload(uint8_t type, const std::string endpoint,
unsigned char *payload, size_t size) {
+ struct coap_context_t* ctx=NULL;
+ struct coap_session_t* session=NULL;
+
+ coap_address_t dst_addr, src_addr;
+ coap_uri_t uri;
+ uri.host.s = reinterpret_cast(const_cast(host_.c_str()));
+ uri.host.length = host_.size();
+ uri.path.s = reinterpret_cast(const_cast(endpoint.c_str()));
+ uri.path.length = endpoint.size();
+ uri.port = port_;
+
+ fd_set readfds;
+ coap_pdu_t* request;
+ unsigned char get_method = 1;
+
+ int res = resolve_address(, _addr.addr.sa);
+ if (res < 0) {
+return CoAPMessage();
+ }
+
+ dst_addr.size = res;
+ dst_addr.addr.sin.sin_port = htons(uri.port);
+
+ void *addrptr = NULL;
+ char port_str[NI_MAXSERV] = "0";
+ char node_str[NI_MAXHOST] = "";
+ switch (dst_addr.addr.sa.sa_family) {
+case AF_INET:
+ addrptr = _addr.addr.sin.sin_addr;
+
+ /* create context for IPv4 */
+ if (!create_session(, , node_str[0] == 0 ? "0.0.0.0" :
node_str, port_str, _addr)) {
+break;
+ } else {
+return CoAPMessage();
+ }
+case AF_INET6:
+ addrptr = _addr.addr.sin6.sin6_addr;
+
+ /* create context for IPv6 */
+ if (!create_session(, , node_str[0] == 0 ? "::" :
node_str, port_str, _addr))
[jira] [Commented] (MINIFICPP-558) Move PayloadSerializer in preparation for Coap
[
https://issues.apache.org/jira/browse/MINIFICPP-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688448#comment-16688448
]
ASF GitHub Bot commented on MINIFICPP-558:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/437#discussion_r233950116
--- Diff: extensions/coap/controllerservice/CoapConnector.cpp ---
@@ -0,0 +1,189 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "CoapConnector.h"
+#include
+#include
+#include
+#include
+#include
+#include "core/Property.h"
+#include "CoapConnector.h"
+#include "io/validation.h"
+#include "properties/Configure.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace controllers {
+
+static core::Property RemoteServer;
+static core::Property Port;
+static core::Property MaxQueueSize;
+
+core::Property
CoapConnectorService::RemoteServer(core::PropertyBuilder::createProperty("Remote
Server")->withDescription("Remote CoAP server")->isRequired(true)->build());
+core::Property
CoapConnectorService::Port(core::PropertyBuilder::createProperty("Remote
Port")->withDescription("Remote CoAP server port")->isRequired(true)->build());
+core::Property
CoapConnectorService::MaxQueueSize(core::PropertyBuilder::createProperty("Max
Queue Size")->withDescription("Max queue size for received data
")->isRequired(true)->build());
+
+void CoapConnectorService::initialize() {
+ if (initialized_)
+return;
+
+ callback_pointers ptrs;
+ ptrs.data_received = receiveMessage;
+ ptrs.received_error = receiveError;
+ init_coap_api(this, );
+
+ std::lock_guard lock(initialization_mutex_);
+
+ ControllerService::initialize();
+
+ initializeProperties();
+
+ initialized_ = true;
+}
+
+void CoapConnectorService::onEnable() {
+ std::string port_str;
+ if (getProperty(RemoteServer.getName(), host_) && !host_.empty() &&
getProperty(Port.getName(), port_str) && !port_str.empty()) {
+core::Property::StringToInt(port_str, port_);
+ } else {
+// this is the case where we aren't being used in the context of a
single controller service.
+if (configuration_->get("nifi.c2.agent.coap.host", host_) &&
configuration_->get("nifi.c2.agent.coap.port", port_str)) {
+ core::Property::StringToInt(port_str, port_);
+}
+
+ }
+}
+
+CoapConnectorService::CoAPMessage
CoapConnectorService::sendPayload(uint8_t type, const std::string endpoint,
unsigned char *payload, size_t size) {
+ struct coap_context_t* ctx=NULL;
+ struct coap_session_t* session=NULL;
+
+ coap_address_t dst_addr, src_addr;
+ coap_uri_t uri;
+ uri.host.s = reinterpret_cast(const_cast(host_.c_str()));
--- End diff --
Ah thanks for the diligence. This was changed last minute ( from coap to
coap2 ) and I may return back to libcoap where the signature is unsigned char.
I posted this early for someone to take a gander from an integration perspective
> Move PayloadSerializer in preparation for Coap
> --
>
> Key: MINIFICPP-558
> URL: https://issues.apache.org/jira/browse/MINIFICPP-558
> Project: NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Mr TheSegfault
>Assignee: Mr TheSegfault
>Priority: Major
> Fix For: 0.6.0
>
>
> Move PayloadSerializer
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5822) ControllerServices not available during @OnScheduled
[ https://issues.apache.org/jira/browse/NIFI-5822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688430#comment-16688430 ] ASF GitHub Bot commented on NIFI-5822: -- Github user bbende commented on the issue: https://github.com/apache/nifi/pull/3171 Reviewing... > ControllerServices not available during @OnScheduled > > > Key: NIFI-5822 > URL: https://issues.apache.org/jira/browse/NIFI-5822 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.8.0 >Reporter: Peter Wicks >Assignee: Mark Payne >Priority: Blocker > Fix For: 1.9.0 > > > If a processor tries to load a ControllerService during `@OnScheduled`, then > `context.getControllerService` contains a null reference to > `controllerServiceProvider`. I traced this back to > `StandardProcessScheduler.startProcessor`, and the reference to > `controllerServiceProvider` is also `null` here. `ExecuteSQL` acquires it's > connection during `@OnScheduled`. > But once we reach `onTrigger`, everything is fine and you can retrieve a > controller service without issue -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-558) Move PayloadSerializer in preparation for Coap
[
https://issues.apache.org/jira/browse/MINIFICPP-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688416#comment-16688416
]
ASF GitHub Bot commented on MINIFICPP-558:
--
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/437#discussion_r233942801
--- Diff: extensions/coap/controllerservice/CoapConnector.cpp ---
@@ -0,0 +1,189 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "CoapConnector.h"
+#include
+#include
+#include
+#include
+#include
+#include "core/Property.h"
+#include "CoapConnector.h"
+#include "io/validation.h"
+#include "properties/Configure.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace controllers {
+
+static core::Property RemoteServer;
+static core::Property Port;
+static core::Property MaxQueueSize;
+
+core::Property
CoapConnectorService::RemoteServer(core::PropertyBuilder::createProperty("Remote
Server")->withDescription("Remote CoAP server")->isRequired(true)->build());
+core::Property
CoapConnectorService::Port(core::PropertyBuilder::createProperty("Remote
Port")->withDescription("Remote CoAP server port")->isRequired(true)->build());
+core::Property
CoapConnectorService::MaxQueueSize(core::PropertyBuilder::createProperty("Max
Queue Size")->withDescription("Max queue size for received data
")->isRequired(true)->build());
+
+void CoapConnectorService::initialize() {
+ if (initialized_)
+return;
+
+ callback_pointers ptrs;
+ ptrs.data_received = receiveMessage;
+ ptrs.received_error = receiveError;
+ init_coap_api(this, );
+
+ std::lock_guard lock(initialization_mutex_);
+
+ ControllerService::initialize();
+
+ initializeProperties();
+
+ initialized_ = true;
+}
+
+void CoapConnectorService::onEnable() {
+ std::string port_str;
+ if (getProperty(RemoteServer.getName(), host_) && !host_.empty() &&
getProperty(Port.getName(), port_str) && !port_str.empty()) {
+core::Property::StringToInt(port_str, port_);
+ } else {
+// this is the case where we aren't being used in the context of a
single controller service.
+if (configuration_->get("nifi.c2.agent.coap.host", host_) &&
configuration_->get("nifi.c2.agent.coap.port", port_str)) {
+ core::Property::StringToInt(port_str, port_);
+}
+
+ }
+}
+
+CoapConnectorService::CoAPMessage
CoapConnectorService::sendPayload(uint8_t type, const std::string endpoint,
unsigned char *payload, size_t size) {
+ struct coap_context_t* ctx=NULL;
+ struct coap_session_t* session=NULL;
+
+ coap_address_t dst_addr, src_addr;
+ coap_uri_t uri;
+ uri.host.s = reinterpret_cast(const_cast(host_.c_str()));
--- End diff --
Uri.host is coap_str_const_t, which has "s" as const member, so there is no
need of const cast here, just reinterpret cast to "const unsigned char *".
> Move PayloadSerializer in preparation for Coap
> --
>
> Key: MINIFICPP-558
> URL: https://issues.apache.org/jira/browse/MINIFICPP-558
> Project: NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Mr TheSegfault
>Assignee: Mr TheSegfault
>Priority: Major
> Fix For: 0.6.0
>
>
> Move PayloadSerializer
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-558) Move PayloadSerializer in preparation for Coap
[
https://issues.apache.org/jira/browse/MINIFICPP-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688414#comment-16688414
]
ASF GitHub Bot commented on MINIFICPP-558:
--
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/437#discussion_r233941448
--- Diff: extensions/coap/controllerservice/CoapConnector.cpp ---
@@ -0,0 +1,189 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "CoapConnector.h"
+#include
+#include
+#include
+#include
+#include
+#include "core/Property.h"
+#include "CoapConnector.h"
+#include "io/validation.h"
+#include "properties/Configure.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace controllers {
+
+static core::Property RemoteServer;
+static core::Property Port;
+static core::Property MaxQueueSize;
+
+core::Property
CoapConnectorService::RemoteServer(core::PropertyBuilder::createProperty("Remote
Server")->withDescription("Remote CoAP server")->isRequired(true)->build());
+core::Property
CoapConnectorService::Port(core::PropertyBuilder::createProperty("Remote
Port")->withDescription("Remote CoAP server port")->isRequired(true)->build());
+core::Property
CoapConnectorService::MaxQueueSize(core::PropertyBuilder::createProperty("Max
Queue Size")->withDescription("Max queue size for received data
")->isRequired(true)->build());
+
+void CoapConnectorService::initialize() {
+ if (initialized_)
+return;
+
+ callback_pointers ptrs;
+ ptrs.data_received = receiveMessage;
+ ptrs.received_error = receiveError;
+ init_coap_api(this, );
+
+ std::lock_guard lock(initialization_mutex_);
+
+ ControllerService::initialize();
+
+ initializeProperties();
+
+ initialized_ = true;
+}
+
+void CoapConnectorService::onEnable() {
+ std::string port_str;
+ if (getProperty(RemoteServer.getName(), host_) && !host_.empty() &&
getProperty(Port.getName(), port_str) && !port_str.empty()) {
+core::Property::StringToInt(port_str, port_);
+ } else {
+// this is the case where we aren't being used in the context of a
single controller service.
+if (configuration_->get("nifi.c2.agent.coap.host", host_) &&
configuration_->get("nifi.c2.agent.coap.port", port_str)) {
+ core::Property::StringToInt(port_str, port_);
+}
+
+ }
+}
+
+CoapConnectorService::CoAPMessage
CoapConnectorService::sendPayload(uint8_t type, const std::string endpoint,
unsigned char *payload, size_t size) {
+ struct coap_context_t* ctx=NULL;
+ struct coap_session_t* session=NULL;
+
+ coap_address_t dst_addr, src_addr;
+ coap_uri_t uri;
+ uri.host.s = reinterpret_cast(const_cast(host_.c_str()));
+ uri.host.length = host_.size();
+ uri.path.s = reinterpret_cast(const_cast(endpoint.c_str()));
+ uri.path.length = endpoint.size();
+ uri.port = port_;
+
+ fd_set readfds;
+ coap_pdu_t* request;
+ unsigned char get_method = 1;
+
+ int res = resolve_address(, _addr.addr.sa);
+ if (res < 0) {
+return CoAPMessage();
+ }
+
+ dst_addr.size = res;
+ dst_addr.addr.sin.sin_port = htons(uri.port);
+
+ void *addrptr = NULL;
+ char port_str[NI_MAXSERV] = "0";
+ char node_str[NI_MAXHOST] = "";
+ switch (dst_addr.addr.sa.sa_family) {
+case AF_INET:
+ addrptr = _addr.addr.sin.sin_addr;
+
+ /* create context for IPv4 */
+ if (!create_session(, , node_str[0] == 0 ? "0.0.0.0" :
node_str, port_str, _addr)) {
+break;
+ } else {
+return CoAPMessage();
+ }
+case AF_INET6:
+ addrptr = _addr.addr.sin6.sin6_addr;
+
+ /* create context for IPv6 */
+ if (!create_session(, , node_str[0] == 0 ? "::" :
node_str, port_str, _addr))
[jira] [Commented] (NIFI-5822) ControllerServices not available during @OnScheduled
[ https://issues.apache.org/jira/browse/NIFI-5822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688398#comment-16688398 ] ASF GitHub Bot commented on NIFI-5822: -- GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/3171 NIFI-5822: Ensure that we don't call FlowController.getControllerServ… …iceProvider() before the ControllerServiceProvider has been initialized Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-5822 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3171.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3171 commit 6aaf3ed25e17523071b3d4e2dd400a43685f873a Author: Mark Payne Date: 2018-11-15T17:33:03Z NIFI-5822: Ensure that we don't call FlowController.getControllerServiceProvider() before the ControllerServiceProvider has been initialized > ControllerServices not available during @OnScheduled > > > Key: NIFI-5822 > URL: https://issues.apache.org/jira/browse/NIFI-5822 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.8.0 >Reporter: Peter Wicks >Priority: Blocker > Fix For: 1.9.0 > > > If a processor tries to load a ControllerService during `@OnScheduled`, then > `context.getControllerService` contains a null reference to > `controllerServiceProvider`. I traced this back to > `StandardProcessScheduler.startProcessor`, and the reference to > `controllerServiceProvider` is also `null` here. `ExecuteSQL` acquires it's > connection during `@OnScheduled`. > But once we reach `onTrigger`, everything is fine and you can retrieve a > controller service without issue -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-667) Create Structural definitions for moving away from C++ classes
[ https://issues.apache.org/jira/browse/MINIFICPP-667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688322#comment-16688322 ] ASF GitHub Bot commented on MINIFICPP-667: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/436#discussion_r233919866 --- Diff: nanofi/include/core/utlist.h --- @@ -0,0 +1,1073 @@ +/* --- End diff -- Okay, fine for me. Happy to approve as it gets moved. > Create Structural definitions for moving away from C++ classes > -- > > Key: MINIFICPP-667 > URL: https://issues.apache.org/jira/browse/MINIFICPP-667 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: nanofi >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > Fix For: 0.6.0 > > > Create Structural definitions for moving away from C++ classes -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-667) Create Structural definitions for moving away from C++ classes
[ https://issues.apache.org/jira/browse/MINIFICPP-667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688321#comment-16688321 ] ASF GitHub Bot commented on MINIFICPP-667: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/436#discussion_r233919353 --- Diff: nanofi/include/core/utlist.h --- @@ -0,0 +1,1073 @@ +/* --- End diff -- sorry CMAKE_SOURCE_DIR. It can go there and should. I think I updated the NOTICE and we direct people to the root source thirdparty, so we can probably just use that. No reason to create levels of thirdparty source dirs yet. > Create Structural definitions for moving away from C++ classes > -- > > Key: MINIFICPP-667 > URL: https://issues.apache.org/jira/browse/MINIFICPP-667 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: nanofi >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > Fix For: 0.6.0 > > > Create Structural definitions for moving away from C++ classes -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-667) Create Structural definitions for moving away from C++ classes
[ https://issues.apache.org/jira/browse/MINIFICPP-667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688299#comment-16688299 ] ASF GitHub Bot commented on MINIFICPP-667: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/436#discussion_r233913016 --- Diff: nanofi/include/core/utlist.h --- @@ -0,0 +1,1073 @@ +/* --- End diff -- Ah thanks for calling this out. I have a more recent commit with a lot of changes, but didn't move this yet and probably would have forgotten. > Create Structural definitions for moving away from C++ classes > -- > > Key: MINIFICPP-667 > URL: https://issues.apache.org/jira/browse/MINIFICPP-667 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: nanofi >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > Fix For: 0.6.0 > > > Create Structural definitions for moving away from C++ classes -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-667) Create Structural definitions for moving away from C++ classes
[
https://issues.apache.org/jira/browse/MINIFICPP-667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688286#comment-16688286
]
ASF GitHub Bot commented on MINIFICPP-667:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/436#discussion_r233910857
--- Diff: nanofi/include/core/cstructs.h ---
@@ -76,6 +80,12 @@ typedef struct {
* ##
*/
+typedef struct{
+ char *key;
--- End diff --
The example code I took this from had this as char*, not sure we can do
this. I'd have to look at the library.
> Create Structural definitions for moving away from C++ classes
> --
>
> Key: MINIFICPP-667
> URL: https://issues.apache.org/jira/browse/MINIFICPP-667
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Affects Versions: nanofi
>Reporter: Mr TheSegfault
>Assignee: Mr TheSegfault
>Priority: Major
> Fix For: 0.6.0
>
>
> Create Structural definitions for moving away from C++ classes
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-667) Create Structural definitions for moving away from C++ classes
[ https://issues.apache.org/jira/browse/MINIFICPP-667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688302#comment-16688302 ] ASF GitHub Bot commented on MINIFICPP-667: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/436#discussion_r233913935 --- Diff: nanofi/include/core/utlist.h --- @@ -0,0 +1,1073 @@ +/* --- End diff -- Not yet in nanofi :) Guess this library is not going to be used in the C++ part. > Create Structural definitions for moving away from C++ classes > -- > > Key: MINIFICPP-667 > URL: https://issues.apache.org/jira/browse/MINIFICPP-667 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: nanofi >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > Fix For: 0.6.0 > > > Create Structural definitions for moving away from C++ classes -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-667) Create Structural definitions for moving away from C++ classes
[ https://issues.apache.org/jira/browse/MINIFICPP-667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688300#comment-16688300 ] ASF GitHub Bot commented on MINIFICPP-667: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/436#discussion_r233913139 --- Diff: nanofi/include/core/utlist.h --- @@ -0,0 +1,1073 @@ +/* --- End diff -- We have a third party dir in the main directory, which is where this should go. > Create Structural definitions for moving away from C++ classes > -- > > Key: MINIFICPP-667 > URL: https://issues.apache.org/jira/browse/MINIFICPP-667 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: nanofi >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > Fix For: 0.6.0 > > > Create Structural definitions for moving away from C++ classes -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-667) Create Structural definitions for moving away from C++ classes
[
https://issues.apache.org/jira/browse/MINIFICPP-667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688292#comment-16688292
]
ASF GitHub Bot commented on MINIFICPP-667:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/436#discussion_r233911949
--- Diff: nanofi/include/core/cstructs.h ---
@@ -76,6 +80,12 @@ typedef struct {
* ##
*/
+typedef struct{
+ char *key;
--- End diff --
I'll add that I'm not against this, but the code pointer's management was a
little different. Again, I'll take a look at if this makes sense. core will be
things that don't reflect a user experience though. That is code that's not
part of our public API.
> Create Structural definitions for moving away from C++ classes
> --
>
> Key: MINIFICPP-667
> URL: https://issues.apache.org/jira/browse/MINIFICPP-667
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Affects Versions: nanofi
>Reporter: Mr TheSegfault
>Assignee: Mr TheSegfault
>Priority: Major
> Fix For: 0.6.0
>
>
> Create Structural definitions for moving away from C++ classes
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-667) Create Structural definitions for moving away from C++ classes
[
https://issues.apache.org/jira/browse/MINIFICPP-667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688290#comment-16688290
]
ASF GitHub Bot commented on MINIFICPP-667:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/436#discussion_r233911441
--- Diff: nanofi/src/api/nanofi.cpp ---
@@ -72,19 +41,17 @@ class DirectoryConfiguration {
* nifi_instance *create_instance(nifi_port const *port) {
*/
nifi_instance *create_instance(const char *url, nifi_port *port) {
- // make sure that we have a thread safe way of initializing the content
directory
- DirectoryConfiguration::initialize();
-
- // need reinterpret cast until we move to C for this module.
- nifi_instance *instance =
reinterpret_cast(malloc(sizeof(nifi_instance)));
+// need reinterpret cast until we move to C for this module.
+ nifi_instance *instance =
(nifi_instance*)(malloc(sizeof(nifi_instance)));
/**
* This API will gradually move away from C++, hence malloc is used for
nifi_instance
* Since minifi::Instance is currently being used, then we need to use
new in that case.
*/
- instance->instance_ptr = new minifi::Instance(url, port->port_id);
+ instance->instance_ptr = create_cxx_instance(url,port);
// may have to translate port ID here in the future
// need reinterpret cast until we move to C for this module.
- instance->port.port_id =
reinterpret_cast(malloc(strlen(port->port_id) + 1));
+ instance->port.port_id = (char*)(malloc(strlen(port->port_id) + 1));
--- End diff --
Yes. We won't be able to use the C++ linter on C files. I haven't pushed
the recent commit, but I'm moving toward renaming this to nanofi.c
> Create Structural definitions for moving away from C++ classes
> --
>
> Key: MINIFICPP-667
> URL: https://issues.apache.org/jira/browse/MINIFICPP-667
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Affects Versions: nanofi
>Reporter: Mr TheSegfault
>Assignee: Mr TheSegfault
>Priority: Major
> Fix For: 0.6.0
>
>
> Create Structural definitions for moving away from C++ classes
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-667) Create Structural definitions for moving away from C++ classes
[
https://issues.apache.org/jira/browse/MINIFICPP-667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688287#comment-16688287
]
ASF GitHub Bot commented on MINIFICPP-667:
--
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/436#discussion_r233910986
--- Diff: nanofi/src/api/nanofi.cpp ---
@@ -72,19 +41,17 @@ class DirectoryConfiguration {
* nifi_instance *create_instance(nifi_port const *port) {
*/
nifi_instance *create_instance(const char *url, nifi_port *port) {
- // make sure that we have a thread safe way of initializing the content
directory
- DirectoryConfiguration::initialize();
-
- // need reinterpret cast until we move to C for this module.
- nifi_instance *instance =
reinterpret_cast(malloc(sizeof(nifi_instance)));
+// need reinterpret cast until we move to C for this module.
+ nifi_instance *instance =
(nifi_instance*)(malloc(sizeof(nifi_instance)));
/**
* This API will gradually move away from C++, hence malloc is used for
nifi_instance
* Since minifi::Instance is currently being used, then we need to use
new in that case.
*/
- instance->instance_ptr = new minifi::Instance(url, port->port_id);
+ instance->instance_ptr = create_cxx_instance(url,port);
// may have to translate port ID here in the future
// need reinterpret cast until we move to C for this module.
- instance->port.port_id =
reinterpret_cast(malloc(strlen(port->port_id) + 1));
+ instance->port.port_id = (char*)(malloc(strlen(port->port_id) + 1));
--- End diff --
I expect this to fail linter, do we already have nanofi out of range of
that?
> Create Structural definitions for moving away from C++ classes
> --
>
> Key: MINIFICPP-667
> URL: https://issues.apache.org/jira/browse/MINIFICPP-667
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Affects Versions: nanofi
>Reporter: Mr TheSegfault
>Assignee: Mr TheSegfault
>Priority: Major
> Fix For: 0.6.0
>
>
> Create Structural definitions for moving away from C++ classes
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-667) Create Structural definitions for moving away from C++ classes
[
https://issues.apache.org/jira/browse/MINIFICPP-667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688284#comment-16688284
]
ASF GitHub Bot commented on MINIFICPP-667:
--
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/436#discussion_r233910142
--- Diff: nanofi/include/core/cstructs.h ---
@@ -76,6 +80,12 @@ typedef struct {
* ##
*/
+typedef struct{
+ char *key;
--- End diff --
Can we have this as const char *?
A bit better user experience if you can write something like
```
key_value.key = "some string"
```
> Create Structural definitions for moving away from C++ classes
> --
>
> Key: MINIFICPP-667
> URL: https://issues.apache.org/jira/browse/MINIFICPP-667
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Affects Versions: nanofi
>Reporter: Mr TheSegfault
>Assignee: Mr TheSegfault
>Priority: Major
> Fix For: 0.6.0
>
>
> Create Structural definitions for moving away from C++ classes
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-667) Create Structural definitions for moving away from C++ classes
[ https://issues.apache.org/jira/browse/MINIFICPP-667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688282#comment-16688282 ] ASF GitHub Bot commented on MINIFICPP-667: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/436#discussion_r233909297 --- Diff: nanofi/include/core/utlist.h --- @@ -0,0 +1,1073 @@ +/* --- End diff -- I would prefer to place this file somewhere else, as it's not really part of core, but a 3rd party library we use. Could you add to a folder that implies this? > Create Structural definitions for moving away from C++ classes > -- > > Key: MINIFICPP-667 > URL: https://issues.apache.org/jira/browse/MINIFICPP-667 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: nanofi >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > Fix For: 0.6.0 > > > Create Structural definitions for moving away from C++ classes -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5788) Introduce batch size limit in PutDatabaseRecord processor
[ https://issues.apache.org/jira/browse/NIFI-5788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688248#comment-16688248 ] ASF GitHub Bot commented on NIFI-5788: -- Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/3128 +1 LGTM, tested with various batch sizes and ran unit tests. Thanks for this improvment! Merged to master > Introduce batch size limit in PutDatabaseRecord processor > - > > Key: NIFI-5788 > URL: https://issues.apache.org/jira/browse/NIFI-5788 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework > Environment: Teradata DB >Reporter: Vadim >Priority: Major > Labels: pull-request-available > Fix For: 1.9.0 > > > Certain JDBC drivers do not support unlimited batch size in INSERT/UPDATE > prepared SQL statements. Specifically, Teradata JDBC driver > ([https://downloads.teradata.com/download/connectivity/jdbc-driver)] would > fail SQL statement when the batch overflows the internal limits. > Dividing data into smaller chunks before the PutDatabaseRecord is applied can > work around the issue in certain scenarios, but generally, this solution is > not perfect because the SQL statements would be executed in different > transaction contexts and data integrity would not be preserved. > The solution suggests the following: > * introduce a new optional parameter in *PutDatabaseRecord* processor, > *max_batch_size* which defines the maximum batch size in INSERT/UPDATE > statement; the default value zero (INFINITY) preserves the old behavior > * divide the input into batches of the specified size and invoke > PreparedStatement.executeBatch() for each batch > Pull request: [https://github.com/apache/nifi/pull/3128] > > [EDIT] Changed batch_size to max_batch_size. The default value would be zero > (INFINITY) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5788) Introduce batch size limit in PutDatabaseRecord processor
[ https://issues.apache.org/jira/browse/NIFI-5788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688243#comment-16688243 ] ASF GitHub Bot commented on NIFI-5788: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/3128 > Introduce batch size limit in PutDatabaseRecord processor > - > > Key: NIFI-5788 > URL: https://issues.apache.org/jira/browse/NIFI-5788 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework > Environment: Teradata DB >Reporter: Vadim >Priority: Major > Labels: pull-request-available > Fix For: 1.9.0 > > > Certain JDBC drivers do not support unlimited batch size in INSERT/UPDATE > prepared SQL statements. Specifically, Teradata JDBC driver > ([https://downloads.teradata.com/download/connectivity/jdbc-driver)] would > fail SQL statement when the batch overflows the internal limits. > Dividing data into smaller chunks before the PutDatabaseRecord is applied can > work around the issue in certain scenarios, but generally, this solution is > not perfect because the SQL statements would be executed in different > transaction contexts and data integrity would not be preserved. > The solution suggests the following: > * introduce a new optional parameter in *PutDatabaseRecord* processor, > *max_batch_size* which defines the maximum batch size in INSERT/UPDATE > statement; the default value zero (INFINITY) preserves the old behavior > * divide the input into batches of the specified size and invoke > PreparedStatement.executeBatch() for each batch > Pull request: [https://github.com/apache/nifi/pull/3128] > > [EDIT] Changed batch_size to max_batch_size. The default value would be zero > (INFINITY) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-676) Cleanup and fix serializable interface implementation
[ https://issues.apache.org/jira/browse/MINIFICPP-676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688124#comment-16688124 ] ASF GitHub Bot commented on MINIFICPP-676: -- Github user arpadboda commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/440 A step forward, there is improvement left (a lot of code duplication exists in stream hierarchy), although this is enough for one PR. > Cleanup and fix serializable interface implementation > - > > Key: MINIFICPP-676 > URL: https://issues.apache.org/jira/browse/MINIFICPP-676 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Minor > > Serializable interface contains a couple of issues: > Type-unsafe template functions > Code duplication > Needless functions > The goal of this ticket is to make the interface cleaner and simpler. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-676) Cleanup and fix serializable interface implementation
[ https://issues.apache.org/jira/browse/MINIFICPP-676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688109#comment-16688109 ] ASF GitHub Bot commented on MINIFICPP-676: -- GitHub user arpadboda opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/440 MINIFICPP-676 - Cleanup and fix serializable interface implementation Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/arpadboda/nifi-minifi-cpp MINIFICPP-676 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/440.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #440 commit cf32a07d7a4b7082a7bc956ea5aa61b343789b81 Author: Arpad Boda Date: 2018-11-14T16:06:17Z MINIFICPP-676 - Cleanup and fix serializable interface implementation > Cleanup and fix serializable interface implementation > - > > Key: MINIFICPP-676 > URL: https://issues.apache.org/jira/browse/MINIFICPP-676 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Minor > > Serializable interface contains a couple of issues: > Type-unsafe template functions > Code duplication > Needless functions > The goal of this ticket is to make the interface cleaner and simpler. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-645) Move from new to malloc in CAPI to facilitate eventual change from C++ to C
[
https://issues.apache.org/jira/browse/MINIFICPP-645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687923#comment-16687923
]
ASF GitHub Bot commented on MINIFICPP-645:
--
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/439#discussion_r233815698
--- Diff: nanofi/src/api/nanofi.cpp ---
@@ -205,7 +205,7 @@ flow_file_record* create_ff_object(const char *file,
const size_t len, const uin
}
flow_file_record* create_ff_object_na(const char *file, const size_t len,
const uint64_t size) {
- flow_file_record *new_ff = new flow_file_record;
+ flow_file_record *new_ff = (flow_file_record*)
malloc(sizeof(flow_file_record));
--- End diff --
Happy newless year! :)
> Move from new to malloc in CAPI to facilitate eventual change from C++ to C
> ---
>
> Key: MINIFICPP-645
> URL: https://issues.apache.org/jira/browse/MINIFICPP-645
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Mr TheSegfault
>Assignee: Arpad Boda
>Priority: Blocker
> Labels: CAPI, nanofi
> Fix For: 0.6.0
>
>
> As gradually move to C we should move out of libminifi and remove the linter.
> Nothing that is returned via the API that is not an opaque pointer should use
> new
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-645) Move from new to malloc in CAPI to facilitate eventual change from C++ to C
[ https://issues.apache.org/jira/browse/MINIFICPP-645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687921#comment-16687921 ] ASF GitHub Bot commented on MINIFICPP-645: -- GitHub user arpadboda opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/439 MINIFICPP-645 - Move from new to malloc in CAPI to facilitate eventua… …l change from C++ to C Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/arpadboda/nifi-minifi-cpp MINIFICPP-645 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/439.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #439 commit 8b0e39dc8ec0dc79c8b3e58fa1a364e736a1fe38 Author: Arpad Boda Date: 2018-11-15T12:00:07Z MINIFICPP-645 - Move from new to malloc in CAPI to facilitate eventual change from C++ to C > Move from new to malloc in CAPI to facilitate eventual change from C++ to C > --- > > Key: MINIFICPP-645 > URL: https://issues.apache.org/jira/browse/MINIFICPP-645 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Mr TheSegfault >Assignee: Arpad Boda >Priority: Blocker > Labels: CAPI, nanofi > Fix For: 0.6.0 > > > As gradually move to C we should move out of libminifi and remove the linter. > Nothing that is returned via the API that is not an opaque pointer should use > new -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5652) LogMessage emits "transfer relationship not specified" error
[
https://issues.apache.org/jira/browse/NIFI-5652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687531#comment-16687531
]
ASF GitHub Bot commented on NIFI-5652:
--
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/3170
> LogMessage emits "transfer relationship not specified" error
>
>
> Key: NIFI-5652
> URL: https://issues.apache.org/jira/browse/NIFI-5652
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Environment: 3 nodes cluster of CentOS 7.4 servers; JRE 1.8
>Reporter: Takeshi Koya
>Assignee: Matt Burgess
>Priority: Minor
> Fix For: 1.9.0
>
>
> Hi, all NiFi developers and users.
> I upgraded our staging cluster from 1.5 to 1.7.1 recently,
> but I am in trouble with LogMessage errors.
> The combination of a log level and a bulletin level that is higher than the
> log level seems to bring about the following "transfer relationship not
> specified" error.
> {code:java}
> 2018-10-02 00:00:00,442 ERROR [Timer-Driven Process Thread-7]
> o.a.nifi.processors.standard.LogMessage LogMessage[id=...] LogMessage[id=...]
> failed to process session due to
> org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=...,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1538373600261-460, container=default,
> section=460], offset=7822560,
> length=8434],offset=0,name=potaufeu_topics_static_ip-172-19-25-78_20181001162829878.tar.gz,size=8434]
> transfer relationship not specified; Processor Administratively Yielded for
> 1 sec: org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=...,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1538373600261-460, container=default,
> section=460], offset=7822560,
> length=8434],offset=0,name=potaufeu_topics_static_ip-172-19-25-78_20181001162829878.tar.gz,size=8434]
> transfer relationship not specified
> {code}
> Errors disappear if we make a bulletin level equal to or lower than a log
> level.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5652) LogMessage emits "transfer relationship not specified" error
[
https://issues.apache.org/jira/browse/NIFI-5652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687528#comment-16687528
]
ASF GitHub Bot commented on NIFI-5652:
--
Github user ijokarumawak commented on the issue:
https://github.com/apache/nifi/pull/3170
LGTM +1, merging. Thanks @mattyb149!
> LogMessage emits "transfer relationship not specified" error
>
>
> Key: NIFI-5652
> URL: https://issues.apache.org/jira/browse/NIFI-5652
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Environment: 3 nodes cluster of CentOS 7.4 servers; JRE 1.8
>Reporter: Takeshi Koya
>Assignee: Matt Burgess
>Priority: Minor
>
> Hi, all NiFi developers and users.
> I upgraded our staging cluster from 1.5 to 1.7.1 recently,
> but I am in trouble with LogMessage errors.
> The combination of a log level and a bulletin level that is higher than the
> log level seems to bring about the following "transfer relationship not
> specified" error.
> {code:java}
> 2018-10-02 00:00:00,442 ERROR [Timer-Driven Process Thread-7]
> o.a.nifi.processors.standard.LogMessage LogMessage[id=...] LogMessage[id=...]
> failed to process session due to
> org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=...,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1538373600261-460, container=default,
> section=460], offset=7822560,
> length=8434],offset=0,name=potaufeu_topics_static_ip-172-19-25-78_20181001162829878.tar.gz,size=8434]
> transfer relationship not specified; Processor Administratively Yielded for
> 1 sec: org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=...,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1538373600261-460, container=default,
> section=460], offset=7822560,
> length=8434],offset=0,name=potaufeu_topics_static_ip-172-19-25-78_20181001162829878.tar.gz,size=8434]
> transfer relationship not specified
> {code}
> Errors disappear if we make a bulletin level equal to or lower than a log
> level.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-675) Parsed headers should not be searched case insensitively
[ https://issues.apache.org/jira/browse/MINIFICPP-675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687144#comment-16687144 ] ASF GitHub Bot commented on MINIFICPP-675: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/438 > Parsed headers should not be searched case insensitively > - > > Key: MINIFICPP-675 > URL: https://issues.apache.org/jira/browse/MINIFICPP-675 > Project: NiFi MiNiFi C++ > Issue Type: Documentation >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > Fix For: 0.6.0 > > > Parsed headers should not be searched case insensitively -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-675) Parsed headers should not be searched case insensitively
[
https://issues.apache.org/jira/browse/MINIFICPP-675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687140#comment-16687140
]
ASF GitHub Bot commented on MINIFICPP-675:
--
Github user apiri commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/438#discussion_r233611389
--- Diff: extensions/http-curl/client/HTTPClient.h ---
@@ -147,6 +146,24 @@ class HTTPClient : public BaseHTTPClient, public
core::Connectable {
return header_response_.header_mapping_;
}
+ /**
+ * Locates the header value ignoring case. This is differente than
returning a mapping
--- End diff --
differente -> different
> Parsed headers should not be searched case insensitively
> -
>
> Key: MINIFICPP-675
> URL: https://issues.apache.org/jira/browse/MINIFICPP-675
> Project: NiFi MiNiFi C++
> Issue Type: Documentation
>Reporter: Mr TheSegfault
>Assignee: Mr TheSegfault
>Priority: Major
>
> Parsed headers should not be searched case insensitively
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-675) Parsed headers should not be searched case insensitively
[
https://issues.apache.org/jira/browse/MINIFICPP-675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687141#comment-16687141
]
ASF GitHub Bot commented on MINIFICPP-675:
--
Github user apiri commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/438#discussion_r233613094
--- Diff: extensions/http-curl/client/HTTPClient.h ---
@@ -147,6 +146,24 @@ class HTTPClient : public BaseHTTPClient, public
core::Connectable {
return header_response_.header_mapping_;
}
+ /**
+ * Locates the header value ignoring case. This is differente than
returning a mapping
+ * of all parsed headers.
+ * This function acknowledges that header entries should searched case
insensitively.
--- End diff --
be searched
> Parsed headers should not be searched case insensitively
> -
>
> Key: MINIFICPP-675
> URL: https://issues.apache.org/jira/browse/MINIFICPP-675
> Project: NiFi MiNiFi C++
> Issue Type: Documentation
>Reporter: Mr TheSegfault
>Assignee: Mr TheSegfault
>Priority: Major
>
> Parsed headers should not be searched case insensitively
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-675) Parsed headers should not be searched case insensitively
[ https://issues.apache.org/jira/browse/MINIFICPP-675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687135#comment-16687135 ] ASF GitHub Bot commented on MINIFICPP-675: -- Github user kevdoran commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/438 Thanks a lot for the quick turn around on this @phrocker! > Parsed headers should not be searched case insensitively > - > > Key: MINIFICPP-675 > URL: https://issues.apache.org/jira/browse/MINIFICPP-675 > Project: NiFi MiNiFi C++ > Issue Type: Documentation >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > > Parsed headers should not be searched case insensitively -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-3229) When a queue contains only Penalized FlowFile's the next processor Tasks/Time statistics becomes extremely large
[ https://issues.apache.org/jira/browse/NIFI-3229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687117#comment-16687117 ] ASF GitHub Bot commented on NIFI-3229: -- Github user patricker commented on the issue: https://github.com/apache/nifi/pull/3131 @markap14 I built unit tests, but I'm having trouble running them at scale. I temporarily checked back in the original method so I could run side-by-side speed comparisons on the same `Connectable`. But if I exceed about 100k tests my unit tests seem to go out to lunch, even if I increase heap so they don't run out. These are checked in right now to run 1 million iterations, but that has not succeeded for me... This is true of the unmodified method if run by itself also (at least on my poor little computer). > When a queue contains only Penalized FlowFile's the next processor Tasks/Time > statistics becomes extremely large > > > Key: NIFI-3229 > URL: https://issues.apache.org/jira/browse/NIFI-3229 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Dmitry Lukyanov >Assignee: Peter Wicks >Priority: Minor > Attachments: flow.xml.gz, nifi-stats.png, nifi-stats2.png > > > fetchfile on `not.found` produces penalized flow file > in this case i'm expecting the next processor will do one task execution when > flow file penalize time over. > but according to stats it executes approximately 1-6 times. > i understand that it could be a feature but stats became really unclear... > maybe there should be two columns? > `All Task/Times` and `Committed Task/Times` -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-675) Parsed headers should not be searched case insensitively
[ https://issues.apache.org/jira/browse/MINIFICPP-675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687089#comment-16687089 ] ASF GitHub Bot commented on MINIFICPP-675: -- Github user apiri commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/438 reviewing > Parsed headers should not be searched case insensitively > - > > Key: MINIFICPP-675 > URL: https://issues.apache.org/jira/browse/MINIFICPP-675 > Project: NiFi MiNiFi C++ > Issue Type: Documentation >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > > Parsed headers should not be searched case insensitively -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5604) Allow GenerateTableFetch to send empty flow files when no rows would be fetched
[
https://issues.apache.org/jira/browse/NIFI-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687049#comment-16687049
]
ASF GitHub Bot commented on NIFI-5604:
--
Github user mattyb149 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3075#discussion_r233598457
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GenerateTableFetch.java
---
@@ -450,7 +450,33 @@ public void onTrigger(final ProcessContext context,
final ProcessSessionFactory
// If there are no SQL statements to be generated, still
output an empty flow file if specified by the user
if (numberOfFetches == 0 &&
outputEmptyFlowFileOnZeroResults) {
-session.transfer((fileToProcess == null) ?
session.create() : session.create(fileToProcess), REL_SUCCESS);
+FlowFile emptyFlowFile = (fileToProcess == null) ?
session.create() : session.create(fileToProcess);
+Map attributesToAdd = new HashMap<>();
+
+attributesToAdd.put("generatetablefetch.tableName",
tableName);
+if (columnNames != null) {
+
attributesToAdd.put("generatetablefetch.columnNames", columnNames);
+}
+whereClause = maxValueClauses.isEmpty() ? "1=1" :
StringUtils.join(maxValueClauses, " AND ");
+if (StringUtils.isNotBlank(whereClause)) {
--- End diff --
Yeah that's probably left over from before we added the "1=1", I'll take a
look and clean it up
> Allow GenerateTableFetch to send empty flow files when no rows would be
> fetched
> ---
>
> Key: NIFI-5604
> URL: https://issues.apache.org/jira/browse/NIFI-5604
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>Priority: Major
>
> Currently, GenerateTableFetch will not output a flow file if there are no
> rows to be fetched. However, it may be desired (especially with incoming flow
> files) that a flow file be sent out even if GTF does not generate any SQL.
> This capability, along with the fragment attributes from NIFI-5601, would
> allow the user to handle this downstream.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-675) Parsed headers should not be searched case insensitively
[
https://issues.apache.org/jira/browse/MINIFICPP-675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687027#comment-16687027
]
ASF GitHub Bot commented on MINIFICPP-675:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/438#discussion_r233590484
--- Diff: extensions/http-curl/tests/HTTPSiteToSiteTests.cpp ---
@@ -123,61 +123,66 @@ struct test_profile {
void run_variance(std::string test_file_location, bool isSecure,
std::string url, const struct test_profile ) {
SiteToSiteTestHarness harness(isSecure);
- SiteToSiteLocationResponder responder(isSecure);
+ SiteToSiteLocationResponder *responder = new
SiteToSiteLocationResponder(isSecure);
--- End diff --
this is a short lived test, we don't care about memory leaks here. and we
don't control stoppage of the web server, so we can avoid issues entirely by
simply adding this to the heap and not concerning ourselves with scope.
> Parsed headers should not be searched case insensitively
> -
>
> Key: MINIFICPP-675
> URL: https://issues.apache.org/jira/browse/MINIFICPP-675
> Project: NiFi MiNiFi C++
> Issue Type: Documentation
>Reporter: Mr TheSegfault
>Assignee: Mr TheSegfault
>Priority: Major
>
> Parsed headers should not be searched case insensitively
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-675) Parsed headers should not be searched case insensitively
[ https://issues.apache.org/jira/browse/MINIFICPP-675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687022#comment-16687022 ] ASF GitHub Bot commented on MINIFICPP-675: -- GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/438 MINIFICPP-675: Fix issue with hearder evaluation and re-enable test MINIFICPP-668: don't append port if it is not valid Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFICPP-675 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/438.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #438 commit ede5ccebee81ef8af51e38828c7afaf7947cf992 Author: Marc Parisi Date: 2018-11-14T19:22:28Z MINIFICPP-675: Fix issue with hearder evaluation and re-enable test MINIFICPP-668: don't append port if it is not valid > Parsed headers should not be searched case insensitively > - > > Key: MINIFICPP-675 > URL: https://issues.apache.org/jira/browse/MINIFICPP-675 > Project: NiFi MiNiFi C++ > Issue Type: Documentation >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > > Parsed headers should not be searched case insensitively -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-665) Fix basestream references
[ https://issues.apache.org/jira/browse/MINIFICPP-665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686897#comment-16686897 ] ASF GitHub Bot commented on MINIFICPP-665: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/435 > Fix basestream references > - > > Key: MINIFICPP-665 > URL: https://issues.apache.org/jira/browse/MINIFICPP-665 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > > At some point basestream was changed to be a composable pointer ( hence the > name base stream ) as a wrapper for a variety of streams. this caused a bug > where a self reference could occur. We overload this stream to be a self > reference since it extends DataStream. This is a very likely usage so we can > avoid the bugs by simply checking if the composition is a reference to self. > This reference to self won't be removed in the interest of time and re-using > Serializable's interface. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5604) Allow GenerateTableFetch to send empty flow files when no rows would be fetched
[
https://issues.apache.org/jira/browse/NIFI-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686841#comment-16686841
]
ASF GitHub Bot commented on NIFI-5604:
--
Github user patricker commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3075#discussion_r233524164
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GenerateTableFetch.java
---
@@ -450,7 +450,33 @@ public void onTrigger(final ProcessContext context,
final ProcessSessionFactory
// If there are no SQL statements to be generated, still
output an empty flow file if specified by the user
if (numberOfFetches == 0 &&
outputEmptyFlowFileOnZeroResults) {
-session.transfer((fileToProcess == null) ?
session.create() : session.create(fileToProcess), REL_SUCCESS);
+FlowFile emptyFlowFile = (fileToProcess == null) ?
session.create() : session.create(fileToProcess);
+Map attributesToAdd = new HashMap<>();
+
+attributesToAdd.put("generatetablefetch.tableName",
tableName);
+if (columnNames != null) {
+
attributesToAdd.put("generatetablefetch.columnNames", columnNames);
+}
+whereClause = maxValueClauses.isEmpty() ? "1=1" :
StringUtils.join(maxValueClauses, " AND ");
+if (StringUtils.isNotBlank(whereClause)) {
--- End diff --
I know it was already like this, but I'm not sure it's even possible for
`whereClause` to be blank based on the logic here. Same for existing code down
below for a normal run.
> Allow GenerateTableFetch to send empty flow files when no rows would be
> fetched
> ---
>
> Key: NIFI-5604
> URL: https://issues.apache.org/jira/browse/NIFI-5604
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>Priority: Major
>
> Currently, GenerateTableFetch will not output a flow file if there are no
> rows to be fetched. However, it may be desired (especially with incoming flow
> files) that a flow file be sent out even if GTF does not generate any SQL.
> This capability, along with the fragment attributes from NIFI-5601, would
> allow the user to handle this downstream.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5604) Allow GenerateTableFetch to send empty flow files when no rows would be fetched
[
https://issues.apache.org/jira/browse/NIFI-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686842#comment-16686842
]
ASF GitHub Bot commented on NIFI-5604:
--
Github user patricker commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3075#discussion_r233524977
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GenerateTableFetch.java
---
@@ -450,7 +450,33 @@ public void onTrigger(final ProcessContext context,
final ProcessSessionFactory
// If there are no SQL statements to be generated, still
output an empty flow file if specified by the user
if (numberOfFetches == 0 &&
outputEmptyFlowFileOnZeroResults) {
-session.transfer((fileToProcess == null) ?
session.create() : session.create(fileToProcess), REL_SUCCESS);
+FlowFile emptyFlowFile = (fileToProcess == null) ?
session.create() : session.create(fileToProcess);
+Map attributesToAdd = new HashMap<>();
+
+attributesToAdd.put("generatetablefetch.tableName",
tableName);
+if (columnNames != null) {
+
attributesToAdd.put("generatetablefetch.columnNames", columnNames);
+}
+whereClause = maxValueClauses.isEmpty() ? "1=1" :
StringUtils.join(maxValueClauses, " AND ");
+if (StringUtils.isNotBlank(whereClause)) {
+
attributesToAdd.put("generatetablefetch.whereClause", whereClause);
+}
+final String maxColumnNames =
StringUtils.join(maxValueColumnNameList, ", ");
+if (StringUtils.isNotBlank(maxColumnNames)) {
+
attributesToAdd.put("generatetablefetch.maxColumnNames", maxColumnNames);
+}
+attributesToAdd.put("generatetablefetch.limit", null);
--- End diff --
There is no existing test in `master`, but I think in normal circumstances
`limit` actually gets written as `"null"`.
```
if ((i == numberOfFetches - 1) && useColumnValsForPaging &&
(maxValueClauses.isEmpty() || customWhereClause != null)) {
maxValueClauses.add(columnForPartitioning + "
<= " + maxValueForPartitioning);
limit = null;
}
```
This value is then written using `String.valueOf(limit)`, which is going to
translate the `null` to `"null"`.
> Allow GenerateTableFetch to send empty flow files when no rows would be
> fetched
> ---
>
> Key: NIFI-5604
> URL: https://issues.apache.org/jira/browse/NIFI-5604
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>Priority: Major
>
> Currently, GenerateTableFetch will not output a flow file if there are no
> rows to be fetched. However, it may be desired (especially with incoming flow
> files) that a flow file be sent out even if GTF does not generate any SQL.
> This capability, along with the fragment attributes from NIFI-5601, would
> allow the user to handle this downstream.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-558) Move PayloadSerializer in preparation for Coap
[
https://issues.apache.org/jira/browse/MINIFICPP-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686769#comment-16686769
]
ASF GitHub Bot commented on MINIFICPP-558:
--
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/437#discussion_r233507600
--- Diff: extensions/coap/protocols/CoapC2Protocol.cpp ---
@@ -0,0 +1,353 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "CoapC2Protocol.h"
+#include "c2/PayloadSerializer.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace c2 {
+
+#include "coap_functions.h"
+#include "io/BaseStream.h"
+
+CoapProtocol::CoapProtocol(std::string name, utils::Identifier uuid)
+: RESTSender(name, uuid),
+ require_registration_(false),
+ logger_(logging::LoggerFactory::getLogger()) {
+}
+
+CoapProtocol::~CoapProtocol() {
+}
+
+void CoapProtocol::initialize(const
std::shared_ptr , const
std::shared_ptr ) {
+ RESTSender::initialize(controller, configure);
+ if (configure->get("nifi.c2.coap.connector.service",
controller_service_name_)) {
+auto service =
controller->getControllerService(controller_service_name_);
+coap_service_ =
std::static_pointer_cast(service);
+ } else {
+logger_->log_info("No CoAP connector configured, so using default
service");
+coap_service_ =
std::make_shared("cs", configure);
+coap_service_->onEnable();
+ }
+}
+
+C2Payload CoapProtocol::consumePayload(const std::string , const
C2Payload , Direction direction, bool async) {
+ return RESTSender::consumePayload(url, payload, direction, false);
+}
+
+int CoapProtocol::writeAcknowledgement(io::BaseStream *stream, const
C2Payload ) {
+ auto ident = payload.getIdentifier();
+ auto state = payload.getStatus().getState();
+ stream->writeUTF(ident);
+ uint8_t payloadState = 0;
+ switch (state) {
+case state::UpdateState::NESTED:
+case state::UpdateState::INITIATE:
+case state::UpdateState::FULLY_APPLIED:
+case state::UpdateState::READ_COMPLETE:
+ payloadState = 0;
+ break;
+case state::UpdateState::NOT_APPLIED:
+case state::UpdateState::PARTIALLY_APPLIED:
+ payloadState = 1;
+ break;
+case state::UpdateState::READ_ERROR:
+ payloadState = 2;
+ break;
+case state::UpdateState::SET_ERROR:
+ payloadState = 3;
+ break;
+ }
+ stream->write(, 1);
+ return 0;
+}
+
+int CoapProtocol::writeHeartbeat(io::BaseStream *stream, const C2Payload
) {
+ bool byte;
+ uint16_t size = 0;
+
+ std::string deviceIdent;
+ // device identifier
+ auto deviceInfo = getPayload("deviceInfo", payload);
+ if (deviceInfo) {
+for (const auto : deviceInfo->getContent()) {
+ if (!getString(, "identifier", )) {
+break;
+ }
+}
+ }
+ stream->writeUTF(deviceIdent, false);
+ std::string agentIdent;
+ // agent identifier
+ auto agentInfo = getPayload("agentInfo", payload);
+ if (agentInfo) {
+for (const auto : agentInfo->getContent()) {
+ if (!getString(, "identifier", )) {
+break;
+ }
+}
+ }
+ if (agentIdent.empty()) {
+return -1;
+ }
+ stream->writeUTF(agentIdent, false);
+
+ auto flowInfo = getPayload("flowInfo", payload);
+
+ if (flowInfo != nullptr) {
+
+auto components = getPayload("components", flowInfo);
+
+auto queues = getPayload("queues", flowInfo);
+
+auto versionedFlowSnapshotURI = getPayload("versionedFlowSnapshotURI",
flowInfo);
+
+if (components && queues && versionedFlowSnapshotURI) {
+ byte = true;
+ stream->write(byte);
+ size =
[jira] [Commented] (MINIFICPP-558) Move PayloadSerializer in preparation for Coap
[
https://issues.apache.org/jira/browse/MINIFICPP-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686758#comment-16686758
]
ASF GitHub Bot commented on MINIFICPP-558:
--
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/437#discussion_r233504886
--- Diff: extensions/coap/protocols/CoapC2Protocol.cpp ---
@@ -0,0 +1,353 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "CoapC2Protocol.h"
+#include "c2/PayloadSerializer.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace c2 {
+
+#include "coap_functions.h"
+#include "io/BaseStream.h"
+
+CoapProtocol::CoapProtocol(std::string name, utils::Identifier uuid)
+: RESTSender(name, uuid),
+ require_registration_(false),
+ logger_(logging::LoggerFactory::getLogger()) {
+}
+
+CoapProtocol::~CoapProtocol() {
+}
+
+void CoapProtocol::initialize(const
std::shared_ptr , const
std::shared_ptr ) {
+ RESTSender::initialize(controller, configure);
+ if (configure->get("nifi.c2.coap.connector.service",
controller_service_name_)) {
+auto service =
controller->getControllerService(controller_service_name_);
+coap_service_ =
std::static_pointer_cast(service);
+ } else {
+logger_->log_info("No CoAP connector configured, so using default
service");
+coap_service_ =
std::make_shared("cs", configure);
+coap_service_->onEnable();
+ }
+}
+
+C2Payload CoapProtocol::consumePayload(const std::string , const
C2Payload , Direction direction, bool async) {
+ return RESTSender::consumePayload(url, payload, direction, false);
+}
+
+int CoapProtocol::writeAcknowledgement(io::BaseStream *stream, const
C2Payload ) {
+ auto ident = payload.getIdentifier();
+ auto state = payload.getStatus().getState();
+ stream->writeUTF(ident);
+ uint8_t payloadState = 0;
+ switch (state) {
+case state::UpdateState::NESTED:
+case state::UpdateState::INITIATE:
+case state::UpdateState::FULLY_APPLIED:
+case state::UpdateState::READ_COMPLETE:
+ payloadState = 0;
+ break;
+case state::UpdateState::NOT_APPLIED:
+case state::UpdateState::PARTIALLY_APPLIED:
+ payloadState = 1;
+ break;
+case state::UpdateState::READ_ERROR:
+ payloadState = 2;
+ break;
+case state::UpdateState::SET_ERROR:
+ payloadState = 3;
+ break;
+ }
+ stream->write(, 1);
+ return 0;
+}
+
+int CoapProtocol::writeHeartbeat(io::BaseStream *stream, const C2Payload
) {
+ bool byte;
+ uint16_t size = 0;
+
+ std::string deviceIdent;
+ // device identifier
+ auto deviceInfo = getPayload("deviceInfo", payload);
+ if (deviceInfo) {
+for (const auto : deviceInfo->getContent()) {
+ if (!getString(, "identifier", )) {
+break;
+ }
+}
+ }
+ stream->writeUTF(deviceIdent, false);
+ std::string agentIdent;
+ // agent identifier
+ auto agentInfo = getPayload("agentInfo", payload);
+ if (agentInfo) {
+for (const auto : agentInfo->getContent()) {
+ if (!getString(, "identifier", )) {
+break;
+ }
+}
+ }
+ if (agentIdent.empty()) {
+return -1;
+ }
+ stream->writeUTF(agentIdent, false);
+
+ auto flowInfo = getPayload("flowInfo", payload);
+
+ if (flowInfo != nullptr) {
+
+auto components = getPayload("components", flowInfo);
+
+auto queues = getPayload("queues", flowInfo);
+
+auto versionedFlowSnapshotURI = getPayload("versionedFlowSnapshotURI",
flowInfo);
+
+if (components && queues && versionedFlowSnapshotURI) {
+ byte = true;
+ stream->write(byte);
+ size =
[jira] [Commented] (NIFI-5652) LogMessage emits "transfer relationship not specified" error
[
https://issues.apache.org/jira/browse/NIFI-5652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686757#comment-16686757
]
ASF GitHub Bot commented on NIFI-5652:
--
GitHub user mattyb149 opened a pull request:
https://github.com/apache/nifi/pull/3170
NIFI-5652: Fixed LogMessage when logging level is disabled
I couldn't write a unit test as I can't change the log level in between
unit tests. I reproduced the issue and verified it is no longer present with a
live NiFi instance running with this patch.
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with NIFI- where is the JIRA number
you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically master)?
- [x] Is your initial contribution a single, squashed commit?
### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file, including the main
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in
which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mattyb149/nifi NIFI-5652
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/3170.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3170
commit 6c08aea9a7394237d5e575086fe965001e3700d7
Author: Matthew Burgess
Date: 2018-11-14T15:50:37Z
NIFI-5652: Fixed LogMessage when logging level is disabled
> LogMessage emits "transfer relationship not specified" error
>
>
> Key: NIFI-5652
> URL: https://issues.apache.org/jira/browse/NIFI-5652
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Environment: 3 nodes cluster of CentOS 7.4 servers; JRE 1.8
>Reporter: Takeshi Koya
>Assignee: Matt Burgess
>Priority: Minor
>
> Hi, all NiFi developers and users.
> I upgraded our staging cluster from 1.5 to 1.7.1 recently,
> but I am in trouble with LogMessage errors.
> The combination of a log level and a bulletin level that is higher than the
> log level seems to bring about the following "transfer relationship not
> specified" error.
> {code:java}
> 2018-10-02 00:00:00,442 ERROR [Timer-Driven Process Thread-7]
> o.a.nifi.processors.standard.LogMessage LogMessage[id=...] LogMessage[id=...]
> failed to process session due to
> org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=...,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1538373600261-460, container=default,
> section=460], offset=7822560,
> length=8434],offset=0,name=potaufeu_topics_static_ip-172-19-25-78_20181001162829878.tar.gz,size=8434]
> transfer relationship not specified; Processor Administratively Yielded for
> 1 sec: org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=...,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1538373600261-460, container=default,
> section=460], offset=7822560,
> length=8434],offset=0,name=potaufeu_topics_static_ip-172-19-25-78_20181001162829878.tar.gz,size=8434]
> transfer relationship not specified
> {code}
> Errors disappear if we make a bulletin level equal to or lower than a log
> level.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-558) Move PayloadSerializer in preparation for Coap
[
https://issues.apache.org/jira/browse/MINIFICPP-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686743#comment-16686743
]
ASF GitHub Bot commented on MINIFICPP-558:
--
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/437#discussion_r233501997
--- Diff: extensions/coap/nanofi/coap_functions.c ---
@@ -0,0 +1,175 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "coap_functions.h"
+
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/**
+ * Initialize the API access. Not thread safe.
+ */
+void init_coap_api(void *rcvr, callback_pointers *ptrs) {
+ global_ptrs.data_received = ptrs->data_received;
+ global_ptrs.received_error = ptrs->received_error;
+ receiver = rcvr;
+}
+
+
+int create_session(coap_context_t **ctx, coap_session_t **session, const
char *node, const char *port, coap_address_t *dst_addr) {
+ int s;
+ struct addrinfo hints;
+ coap_proto_t proto = COAP_PROTO_UDP;
+ struct addrinfo *result, *rp;
+
+ memset(, 0, sizeof(struct addrinfo));
+ hints.ai_family = AF_UNSPEC; // ipv4 or ipv6
+ hints.ai_socktype = COAP_PROTO_RELIABLE(proto) ? SOCK_STREAM :
SOCK_DGRAM;
+ hints.ai_flags = AI_PASSIVE | AI_NUMERICHOST | AI_NUMERICSERV | AI_ALL;
+
+ s = getaddrinfo(node, port, , );
+ if (s != 0) {
+fprintf(stderr, "getaddrinfo: %s\n", gai_strerror(s));
+return -1;
+ }
+
+ for (rp = result; rp != NULL; rp = rp->ai_next) {
+coap_address_t addr;
+
+if (rp->ai_addrlen <= sizeof(addr.addr)) {
+ coap_address_init();
+ addr.size = rp->ai_addrlen;
+ memcpy(, rp->ai_addr, rp->ai_addrlen);
+
+ *ctx = coap_new_context(0x00);
+
+ *session = coap_new_client_session(*ctx, , dst_addr, proto);
+ if (*ctx && *session) {
+freeaddrinfo(result);
+return 0;
+ }
+}
+ }
+
+ fprintf(stderr, "no context available for interface '%s'\n", node);
+
+ freeaddrinfo(result);
+ return -1;
+}
+
+struct coap_pdu_t *create_request(struct coap_context_t *ctx,struct
coap_session_t *session,coap_optlist_t **optlist, unsigned char code,
coap_str_const_t *ptr) {
+ coap_pdu_t *pdu;
+
+ if (!(pdu = coap_new_pdu(session)))
+return NULL;
+
+ pdu->type = COAP_MESSAGE_CON;
+ pdu->tid = coap_new_message_id(session);
+ pdu->code = code;
+
+ if (optlist){
+coap_add_optlist_pdu(pdu, optlist);
+ }
+
+ int flags = 0;
+ coap_add_data(pdu, ptr->length, ptr->s);
+ return pdu;
+}
+
+int coap_event(struct coap_context_t *ctx, coap_event_t event, struct
coap_session_t *session){
+if (event == COAP_EVENT_SESSION_FAILED && global_ptrs.received_error){
+global_ptrs.received_error(receiver, ctx, -1);
+}
+return 0;
+}
+
+void no_acknowledgement(struct coap_context_t *ctx, coap_session_t
*session, coap_pdu_t *sent, coap_nack_reason_t reason, const coap_tid_t id){
+ if (global_ptrs.received_error){
+ global_ptrs.received_error(receiver, ctx, -1);
+ }
+}
+
+void response_handler(struct coap_context_t *ctx, struct coap_session_t
*session, coap_pdu_t *sent, coap_pdu_t *received, const coap_tid_t id) {
+ unsigned char* data;
+ size_t data_len;
+ coap_opt_iterator_t opt_iter;
+ coap_opt_t * block_opt = coap_check_option(received, COAP_OPTION_BLOCK1,
_iter);
+ if (block_opt) {
+ } else {
+if (!global_ptrs.data_received){
+ return;
+}
+
+if (COAP_RESPONSE_CLASS(received->code) == 2 || received->code ==
COAP_RESPONSE_400) {
+ if (coap_get_data(received, _len, )) {
+if (global_ptrs.data_received)
+global_ptrs.data_received(receiver, ctx, received->code, data,
_len);
+ }
+}
[jira] [Commented] (MINIFICPP-558) Move PayloadSerializer in preparation for Coap
[
https://issues.apache.org/jira/browse/MINIFICPP-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686739#comment-16686739
]
ASF GitHub Bot commented on MINIFICPP-558:
--
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/437#discussion_r233500950
--- Diff: extensions/coap/nanofi/coap_functions.c ---
@@ -0,0 +1,175 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "coap_functions.h"
+
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/**
+ * Initialize the API access. Not thread safe.
+ */
+void init_coap_api(void *rcvr, callback_pointers *ptrs) {
+ global_ptrs.data_received = ptrs->data_received;
+ global_ptrs.received_error = ptrs->received_error;
+ receiver = rcvr;
+}
+
+
+int create_session(coap_context_t **ctx, coap_session_t **session, const
char *node, const char *port, coap_address_t *dst_addr) {
+ int s;
+ struct addrinfo hints;
+ coap_proto_t proto = COAP_PROTO_UDP;
+ struct addrinfo *result, *rp;
+
+ memset(, 0, sizeof(struct addrinfo));
+ hints.ai_family = AF_UNSPEC; // ipv4 or ipv6
+ hints.ai_socktype = COAP_PROTO_RELIABLE(proto) ? SOCK_STREAM :
SOCK_DGRAM;
+ hints.ai_flags = AI_PASSIVE | AI_NUMERICHOST | AI_NUMERICSERV | AI_ALL;
+
+ s = getaddrinfo(node, port, , );
+ if (s != 0) {
+fprintf(stderr, "getaddrinfo: %s\n", gai_strerror(s));
+return -1;
+ }
+
+ for (rp = result; rp != NULL; rp = rp->ai_next) {
+coap_address_t addr;
+
+if (rp->ai_addrlen <= sizeof(addr.addr)) {
+ coap_address_init();
+ addr.size = rp->ai_addrlen;
+ memcpy(, rp->ai_addr, rp->ai_addrlen);
+
+ *ctx = coap_new_context(0x00);
+
+ *session = coap_new_client_session(*ctx, , dst_addr, proto);
+ if (*ctx && *session) {
+freeaddrinfo(result);
+return 0;
+ }
+}
+ }
+
+ fprintf(stderr, "no context available for interface '%s'\n", node);
+
+ freeaddrinfo(result);
+ return -1;
+}
+
+struct coap_pdu_t *create_request(struct coap_context_t *ctx,struct
coap_session_t *session,coap_optlist_t **optlist, unsigned char code,
coap_str_const_t *ptr) {
+ coap_pdu_t *pdu;
+
+ if (!(pdu = coap_new_pdu(session)))
+return NULL;
+
+ pdu->type = COAP_MESSAGE_CON;
+ pdu->tid = coap_new_message_id(session);
+ pdu->code = code;
+
+ if (optlist){
+coap_add_optlist_pdu(pdu, optlist);
+ }
+
+ int flags = 0;
+ coap_add_data(pdu, ptr->length, ptr->s);
+ return pdu;
+}
+
+int coap_event(struct coap_context_t *ctx, coap_event_t event, struct
coap_session_t *session){
+if (event == COAP_EVENT_SESSION_FAILED && global_ptrs.received_error){
+global_ptrs.received_error(receiver, ctx, -1);
+}
+return 0;
+}
+
+void no_acknowledgement(struct coap_context_t *ctx, coap_session_t
*session, coap_pdu_t *sent, coap_nack_reason_t reason, const coap_tid_t id){
+ if (global_ptrs.received_error){
+ global_ptrs.received_error(receiver, ctx, -1);
+ }
+}
+
+void response_handler(struct coap_context_t *ctx, struct coap_session_t
*session, coap_pdu_t *sent, coap_pdu_t *received, const coap_tid_t id) {
+ unsigned char* data;
+ size_t data_len;
+ coap_opt_iterator_t opt_iter;
+ coap_opt_t * block_opt = coap_check_option(received, COAP_OPTION_BLOCK1,
_iter);
+ if (block_opt) {
--- End diff --
Why not
```
if(!block_opt)
```
?
> Move PayloadSerializer in preparation for Coap
> --
>
> Key: MINIFICPP-558
> URL: https://issues.apache.org/jira/browse/MINIFICPP-558
> Project: NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Mr TheSegfault
[jira] [Commented] (MINIFICPP-558) Move PayloadSerializer in preparation for Coap
[
https://issues.apache.org/jira/browse/MINIFICPP-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686737#comment-16686737
]
ASF GitHub Bot commented on MINIFICPP-558:
--
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/437#discussion_r233500619
--- Diff: extensions/coap/nanofi/coap_functions.c ---
@@ -0,0 +1,175 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "coap_functions.h"
+
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/**
+ * Initialize the API access. Not thread safe.
+ */
+void init_coap_api(void *rcvr, callback_pointers *ptrs) {
+ global_ptrs.data_received = ptrs->data_received;
+ global_ptrs.received_error = ptrs->received_error;
+ receiver = rcvr;
+}
+
+
+int create_session(coap_context_t **ctx, coap_session_t **session, const
char *node, const char *port, coap_address_t *dst_addr) {
+ int s;
+ struct addrinfo hints;
+ coap_proto_t proto = COAP_PROTO_UDP;
+ struct addrinfo *result, *rp;
+
+ memset(, 0, sizeof(struct addrinfo));
+ hints.ai_family = AF_UNSPEC; // ipv4 or ipv6
+ hints.ai_socktype = COAP_PROTO_RELIABLE(proto) ? SOCK_STREAM :
SOCK_DGRAM;
+ hints.ai_flags = AI_PASSIVE | AI_NUMERICHOST | AI_NUMERICSERV | AI_ALL;
+
+ s = getaddrinfo(node, port, , );
+ if (s != 0) {
+fprintf(stderr, "getaddrinfo: %s\n", gai_strerror(s));
+return -1;
+ }
+
+ for (rp = result; rp != NULL; rp = rp->ai_next) {
+coap_address_t addr;
+
+if (rp->ai_addrlen <= sizeof(addr.addr)) {
+ coap_address_init();
+ addr.size = rp->ai_addrlen;
+ memcpy(, rp->ai_addr, rp->ai_addrlen);
+
+ *ctx = coap_new_context(0x00);
+
+ *session = coap_new_client_session(*ctx, , dst_addr, proto);
+ if (*ctx && *session) {
+freeaddrinfo(result);
+return 0;
+ }
+}
+ }
+
+ fprintf(stderr, "no context available for interface '%s'\n", node);
+
+ freeaddrinfo(result);
+ return -1;
+}
+
+struct coap_pdu_t *create_request(struct coap_context_t *ctx,struct
coap_session_t *session,coap_optlist_t **optlist, unsigned char code,
coap_str_const_t *ptr) {
+ coap_pdu_t *pdu;
+
+ if (!(pdu = coap_new_pdu(session)))
+return NULL;
+
+ pdu->type = COAP_MESSAGE_CON;
+ pdu->tid = coap_new_message_id(session);
+ pdu->code = code;
+
+ if (optlist){
+coap_add_optlist_pdu(pdu, optlist);
+ }
+
+ int flags = 0;
+ coap_add_data(pdu, ptr->length, ptr->s);
+ return pdu;
+}
+
+int coap_event(struct coap_context_t *ctx, coap_event_t event, struct
coap_session_t *session){
+if (event == COAP_EVENT_SESSION_FAILED && global_ptrs.received_error){
--- End diff --
Please indent with 2 spaces
> Move PayloadSerializer in preparation for Coap
> --
>
> Key: MINIFICPP-558
> URL: https://issues.apache.org/jira/browse/MINIFICPP-558
> Project: NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Mr TheSegfault
>Assignee: Mr TheSegfault
>Priority: Major
> Fix For: 0.6.0
>
>
> Move PayloadSerializer
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-558) Move PayloadSerializer in preparation for Coap
[
https://issues.apache.org/jira/browse/MINIFICPP-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686735#comment-16686735
]
ASF GitHub Bot commented on MINIFICPP-558:
--
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/437#discussion_r233500462
--- Diff: extensions/coap/nanofi/coap_functions.c ---
@@ -0,0 +1,175 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "coap_functions.h"
+
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/**
+ * Initialize the API access. Not thread safe.
+ */
+void init_coap_api(void *rcvr, callback_pointers *ptrs) {
+ global_ptrs.data_received = ptrs->data_received;
+ global_ptrs.received_error = ptrs->received_error;
+ receiver = rcvr;
+}
+
+
+int create_session(coap_context_t **ctx, coap_session_t **session, const
char *node, const char *port, coap_address_t *dst_addr) {
+ int s;
+ struct addrinfo hints;
+ coap_proto_t proto = COAP_PROTO_UDP;
+ struct addrinfo *result, *rp;
+
+ memset(, 0, sizeof(struct addrinfo));
+ hints.ai_family = AF_UNSPEC; // ipv4 or ipv6
+ hints.ai_socktype = COAP_PROTO_RELIABLE(proto) ? SOCK_STREAM :
SOCK_DGRAM;
+ hints.ai_flags = AI_PASSIVE | AI_NUMERICHOST | AI_NUMERICSERV | AI_ALL;
+
+ s = getaddrinfo(node, port, , );
+ if (s != 0) {
+fprintf(stderr, "getaddrinfo: %s\n", gai_strerror(s));
+return -1;
+ }
+
+ for (rp = result; rp != NULL; rp = rp->ai_next) {
+coap_address_t addr;
+
+if (rp->ai_addrlen <= sizeof(addr.addr)) {
+ coap_address_init();
+ addr.size = rp->ai_addrlen;
+ memcpy(, rp->ai_addr, rp->ai_addrlen);
+
+ *ctx = coap_new_context(0x00);
+
+ *session = coap_new_client_session(*ctx, , dst_addr, proto);
+ if (*ctx && *session) {
+freeaddrinfo(result);
+return 0;
+ }
+}
+ }
+
+ fprintf(stderr, "no context available for interface '%s'\n", node);
+
+ freeaddrinfo(result);
+ return -1;
+}
+
+struct coap_pdu_t *create_request(struct coap_context_t *ctx,struct
coap_session_t *session,coap_optlist_t **optlist, unsigned char code,
coap_str_const_t *ptr) {
+ coap_pdu_t *pdu;
+
+ if (!(pdu = coap_new_pdu(session)))
+return NULL;
+
+ pdu->type = COAP_MESSAGE_CON;
+ pdu->tid = coap_new_message_id(session);
+ pdu->code = code;
+
+ if (optlist){
+coap_add_optlist_pdu(pdu, optlist);
+ }
+
+ int flags = 0;
+ coap_add_data(pdu, ptr->length, ptr->s);
+ return pdu;
+}
+
+int coap_event(struct coap_context_t *ctx, coap_event_t event, struct
coap_session_t *session){
+if (event == COAP_EVENT_SESSION_FAILED && global_ptrs.received_error){
+global_ptrs.received_error(receiver, ctx, -1);
+}
+return 0;
+}
+
+void no_acknowledgement(struct coap_context_t *ctx, coap_session_t
*session, coap_pdu_t *sent, coap_nack_reason_t reason, const coap_tid_t id){
+ if (global_ptrs.received_error){
+ global_ptrs.received_error(receiver, ctx, -1);
--- End diff --
Indentation
> Move PayloadSerializer in preparation for Coap
> --
>
> Key: MINIFICPP-558
> URL: https://issues.apache.org/jira/browse/MINIFICPP-558
> Project: NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Mr TheSegfault
>Assignee: Mr TheSegfault
>Priority: Major
> Fix For: 0.6.0
>
>
> Move PayloadSerializer
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-558) Move PayloadSerializer in preparation for Coap
[
https://issues.apache.org/jira/browse/MINIFICPP-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686730#comment-16686730
]
ASF GitHub Bot commented on MINIFICPP-558:
--
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/437#discussion_r233499744
--- Diff: extensions/coap/controllerservice/CoapConnector.h ---
@@ -0,0 +1,207 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef LIBMINIFI_INCLUDE_CONTROLLERS_COAPCONNECTOR_H_
+#define LIBMINIFI_INCLUDE_CONTROLLERS_COAPCONNECTOR_H_
+
+
+#include "core/logging/LoggerConfiguration.h"
+#include "coap_functions.h"
+#include "core/controller/ControllerService.h"
+#include
+#include
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace controllers {
+
+/**
+ * Purpose and Justification: Controller services function as a layerable
way to provide
+ * services to internal services. While a controller service is generally
configured from the flow,
+ * we want to follow the open closed principle and provide CoAP services
to other components.
+ *
+ *
+ */
+class CoapConnectorService : public core::controller::ControllerService {
+ public:
+
+ /**
+ * CoapMessage is in internal message format that is sent to and from
consumers of this controller service.
+ */
+ class CoAPMessage {
+ public:
+
+explicit CoAPMessage(unsigned int code = 0)
+: code_(code) {
+}
+
+explicit CoAPMessage(unsigned int code, unsigned char *data, size_t
dataLen)
+: code_(code) {
+ if (data && dataLen > 0)
+std::copy(data, data + dataLen, std::back_inserter(data_));
+}
+
+CoAPMessage(const CoAPMessage ) = delete;
+
+CoAPMessage(CoAPMessage &) = default;
+
+~CoAPMessage() {
+}
+
+size_t getSize() const {
+ return data_.size();
+}
+unsigned char const *getData() const {
+ return data_.data();
+}
+
+bool isRegistrationRequest() {
+ if (data_.size() != 8) {
+return false;
+ }
+ return code_ == COAP_RESPONSE_400 && std::string((char*)
data_.data(), data_.size()) == "register";
+}
+CoAPMessage =(const CoAPMessage ) = delete;
+CoAPMessage =(CoAPMessage &) = default;
+ private:
+unsigned int code_;
+std::vector data_;
+ };
+
+ /**
+ * Constructors for the controller service.
+ */
+ explicit CoapConnectorService(const std::string , const std::string
)
+ : ControllerService(name, id),
+port_(0),
+initialized_(false),
+logger_(logging::LoggerFactory::getLogger())
{
+initialize();
+ }
+
+ explicit CoapConnectorService(const std::string , utils::Identifier
uuid = utils::Identifier())
+ : ControllerService(name, uuid),
+port_(0),
+initialized_(false),
+logger_(logging::LoggerFactory::getLogger())
{
+initialize();
+ }
+
+ explicit CoapConnectorService(const std::string , const
std::shared_ptr )
+ : ControllerService(name),
+port_(0),
+initialized_(false),
+logger_(logging::LoggerFactory::getLogger())
{
+setConfiguration(configuration);
+initialize();
+ }
+
+ /**
+ * Parameters needed.
+ */
+ static core::Property RemoteServer;
+ static core::Property Port;
+ static core::Property MaxQueueSize;
+
+ virtual void initialize();
+
+ void yield() {
+
+ }
+
+ bool isRunning() {
+return getState() == core::controller::ControllerServiceState::ENABLED;
+ }
+
+ bool isWorkAvailable() {
+return false;
+ }
+
+ virtual void onEnable();
+
+ /**
[jira] [Commented] (MINIFICPP-558) Move PayloadSerializer in preparation for Coap
[ https://issues.apache.org/jira/browse/MINIFICPP-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686721#comment-16686721 ] ASF GitHub Bot commented on MINIFICPP-558: -- Github user arpadboda commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/437#discussion_r233497401 --- Diff: docker/test/integration/minifi/test/__init__.py --- @@ -42,6 +42,8 @@ def __init__(self, output_validator): self.segfault = False +self.segfault = False --- End diff -- This is already there two lines above. > Move PayloadSerializer in preparation for Coap > -- > > Key: MINIFICPP-558 > URL: https://issues.apache.org/jira/browse/MINIFICPP-558 > Project: NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > Fix For: 0.6.0 > > > Move PayloadSerializer -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-665) Fix basestream references
[
https://issues.apache.org/jira/browse/MINIFICPP-665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686699#comment-16686699
]
ASF GitHub Bot commented on MINIFICPP-665:
--
Github user arpadboda commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/435#discussion_r232960242
--- Diff: libminifi/src/io/BaseStream.cpp ---
@@ -143,15 +147,19 @@ int BaseStream::read(uint8_t *value, int len) {
* @param buflen
*/
int BaseStream::readData(std::vector , int buflen) {
- return Serializable::read([0], buflen,
reinterpret_cast(composable_stream_));
+ return Serializable::read([0], buflen, composable_stream_);
}
/**
* Reads data and places it into buf
* @param buf buffer in which we extract data
* @param buflen
*/
int BaseStream::readData(uint8_t *buf, int buflen) {
- return Serializable::read(buf, buflen,
reinterpret_cast(composable_stream_));
+ if (composable_stream_ == this) {
--- End diff --
Likely?
> Fix basestream references
> -
>
> Key: MINIFICPP-665
> URL: https://issues.apache.org/jira/browse/MINIFICPP-665
> Project: NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Mr TheSegfault
>Assignee: Mr TheSegfault
>Priority: Major
>
> At some point basestream was changed to be a composable pointer ( hence the
> name base stream ) as a wrapper for a variety of streams. this caused a bug
> where a self reference could occur. We overload this stream to be a self
> reference since it extends DataStream. This is a very likely usage so we can
> avoid the bugs by simply checking if the composition is a reference to self.
> This reference to self won't be removed in the interest of time and re-using
> Serializable's interface.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4130) TransformXml - provide a way to define XSLT without external files
[ https://issues.apache.org/jira/browse/NIFI-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686617#comment-16686617 ] ASF GitHub Bot commented on NIFI-4130: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1953 > TransformXml - provide a way to define XSLT without external files > -- > > Key: NIFI-4130 > URL: https://issues.apache.org/jira/browse/NIFI-4130 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Fix For: 1.9.0 > > > In cluster deployments the need to reference external configuration files can > be annoying since it requires to access to all the NiFi nodes and to > correctly deploy the files. It would be interesting to leverage the lookup > controller services in TransformXml to provide a way to define XSLT directly > from the UI without external configuration files. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4130) TransformXml - provide a way to define XSLT without external files
[ https://issues.apache.org/jira/browse/NIFI-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686614#comment-16686614 ] ASF GitHub Bot commented on NIFI-4130: -- Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/1953 +1 LGTM, thanks for the review @bdesert and the improvement @pvillard31 ! Merging to master > TransformXml - provide a way to define XSLT without external files > -- > > Key: NIFI-4130 > URL: https://issues.apache.org/jira/browse/NIFI-4130 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Fix For: 1.9.0 > > > In cluster deployments the need to reference external configuration files can > be annoying since it requires to access to all the NiFi nodes and to > correctly deploy the files. It would be interesting to leverage the lookup > controller services in TransformXml to provide a way to define XSLT directly > from the UI without external configuration files. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-558) Move PayloadSerializer in preparation for Coap
[
https://issues.apache.org/jira/browse/MINIFICPP-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686605#comment-16686605
]
ASF GitHub Bot commented on MINIFICPP-558:
--
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/437#discussion_r233474220
--- Diff: extensions/coap/CMakeLists.txt ---
@@ -0,0 +1,92 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+
+include(${CMAKE_SOURCE_DIR}/extensions/ExtensionHeader.txt)
+include_directories(protocols nanofi controllerservice)
+include_directories(../http-curl/)
+
+file(GLOB CSOURCES "nanofi/*.c")
+file(GLOB SOURCES "*.cpp" "protocols/*.cpp" "processors/*.cpp"
"controllerservice/*.cpp" )
+
+add_library(nanofi-coap-c STATIC ${CSOURCES})
+add_library(minifi-coap STATIC ${SOURCES})
+set_property(TARGET minifi-coap PROPERTY POSITION_INDEPENDENT_CODE ON)
+
+if(CMAKE_THREAD_LIBS_INIT)
+ target_link_libraries(minifi-coap "${CMAKE_THREAD_LIBS_INIT}")
+endif()
+
+ set(BASE_DIR "${CMAKE_CURRENT_BINARY_DIR}/extensions/coap")
+ if (APPLE)
+ set(BYPRODUCT
"${BASE_DIR}/extensions/coap/thirdparty/libcoap-src/.libs/libcoap-2-gnutls.a")
--- End diff --
this may need a little modification as artifacts can vary depending on what
is used for building TLS
> Move PayloadSerializer in preparation for Coap
> --
>
> Key: MINIFICPP-558
> URL: https://issues.apache.org/jira/browse/MINIFICPP-558
> Project: NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Mr TheSegfault
>Assignee: Mr TheSegfault
>Priority: Major
> Fix For: 0.6.0
>
>
> Move PayloadSerializer
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-558) Move PayloadSerializer in preparation for Coap
[ https://issues.apache.org/jira/browse/MINIFICPP-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686602#comment-16686602 ] ASF GitHub Bot commented on MINIFICPP-558: -- Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/437 Adding tests presently, but providing a wip view. > Move PayloadSerializer in preparation for Coap > -- > > Key: MINIFICPP-558 > URL: https://issues.apache.org/jira/browse/MINIFICPP-558 > Project: NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > Fix For: 0.6.0 > > > Move PayloadSerializer -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-558) Move PayloadSerializer in preparation for Coap
[ https://issues.apache.org/jira/browse/MINIFICPP-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686601#comment-16686601 ] ASF GitHub Bot commented on MINIFICPP-558: -- GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/437 MINIFICPP-558: initial provisioning for CoAP Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFICPP-558 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/437.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #437 commit bee4aad6c91b1926e90ac9ce5646e04e865410cc Author: Marc Parisi Date: 2018-10-23T15:51:19Z MINIFICPP-558: initial provisioning for CoAP > Move PayloadSerializer in preparation for Coap > -- > > Key: MINIFICPP-558 > URL: https://issues.apache.org/jira/browse/MINIFICPP-558 > Project: NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > Fix For: 0.6.0 > > > Move PayloadSerializer -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4130) TransformXml - provide a way to define XSLT without external files
[ https://issues.apache.org/jira/browse/NIFI-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686562#comment-16686562 ] ASF GitHub Bot commented on NIFI-4130: -- Github user bdesert commented on the issue: https://github.com/apache/nifi/pull/1953 +1 LGTM. Tested on local env, with XSLT as a file (regression), with lookup service, cache size 0 and >0, all works as expected. Ready for merge. @mattyb149 , please could you please give a final look? > TransformXml - provide a way to define XSLT without external files > -- > > Key: NIFI-4130 > URL: https://issues.apache.org/jira/browse/NIFI-4130 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > > In cluster deployments the need to reference external configuration files can > be annoying since it requires to access to all the NiFi nodes and to > correctly deploy the files. It would be interesting to leverage the lookup > controller services in TransformXml to provide a way to define XSLT directly > from the UI without external configuration files. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4130) TransformXml - provide a way to define XSLT without external files
[ https://issues.apache.org/jira/browse/NIFI-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686212#comment-16686212 ] ASF GitHub Bot commented on NIFI-4130: -- Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/1953 Hey @bdesert - thanks a lot for the review, I just pushed a commit that should address your comments. > TransformXml - provide a way to define XSLT without external files > -- > > Key: NIFI-4130 > URL: https://issues.apache.org/jira/browse/NIFI-4130 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > > In cluster deployments the need to reference external configuration files can > be annoying since it requires to access to all the NiFi nodes and to > correctly deploy the files. It would be interesting to leverage the lookup > controller services in TransformXml to provide a way to define XSLT directly > from the UI without external configuration files. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5815) PutORC processor "Restricted" still requires access to restricted components regardless of restriction
[ https://issues.apache.org/jira/browse/NIFI-5815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686084#comment-16686084 ] ASF GitHub Bot commented on NIFI-5815: -- Github user ijokarumawak commented on the issue: https://github.com/apache/nifi/pull/3169 LGTM +1, merging. Thanks @pvillard31! > PutORC processor "Restricted" still requires access to restricted components > regardless of restriction > -- > > Key: NIFI-5815 > URL: https://issues.apache.org/jira/browse/NIFI-5815 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.6.0, 1.7.0, 1.8.0, 1.7.1 >Reporter: Matthew Clarke >Assignee: Pierre Villard >Priority: Major > > When the new more granular policies were introduced for the "Access > Restricted Components" access policy, the PutORC processor was missed. It > still requires that users have full access to all restricted components in > order to use this processor. This has side affect of any user who needs to > use this components having access to all components in every sub policy of > restricted-components. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5815) PutORC processor "Restricted" still requires access to restricted components regardless of restriction
[ https://issues.apache.org/jira/browse/NIFI-5815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686086#comment-16686086 ] ASF GitHub Bot commented on NIFI-5815: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/3169 > PutORC processor "Restricted" still requires access to restricted components > regardless of restriction > -- > > Key: NIFI-5815 > URL: https://issues.apache.org/jira/browse/NIFI-5815 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.6.0, 1.7.0, 1.8.0, 1.7.1 >Reporter: Matthew Clarke >Assignee: Pierre Villard >Priority: Major > > When the new more granular policies were introduced for the "Access > Restricted Components" access policy, the PutORC processor was missed. It > still requires that users have full access to all restricted components in > order to use this processor. This has side affect of any user who needs to > use this components having access to all components in every sub policy of > restricted-components. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5604) Allow GenerateTableFetch to send empty flow files when no rows would be fetched
[
https://issues.apache.org/jira/browse/NIFI-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685496#comment-16685496
]
ASF GitHub Bot commented on NIFI-5604:
--
Github user mattyb149 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3075#discussion_r233142376
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GenerateTableFetch.java
---
@@ -434,48 +448,53 @@ public void onTrigger(final ProcessContext context,
final ProcessSessionFactory
numberOfFetches = (partitionSize == 0) ? 1 : (rowCount
/ partitionSize) + (rowCount % partitionSize == 0 ? 0 : 1);
}
-// Generate SQL statements to read "pages" of data
-Long limit = partitionSize == 0 ? null : (long)
partitionSize;
-final String fragmentIdentifier =
UUID.randomUUID().toString();
-for (long i = 0; i < numberOfFetches; i++) {
-// Add a right bounding for the partitioning column if
necessary (only on last partition, meaning we don't need the limit)
-if ((i == numberOfFetches - 1) &&
useColumnValsForPaging && (maxValueClauses.isEmpty() || customWhereClause !=
null)) {
-maxValueClauses.add(columnForPartitioning + " <= "
+ maxValueForPartitioning);
-limit = null;
-}
+// If there are no SQL statements to be generated, still
output an empty flow file if specified by the user
+if (numberOfFetches == 0 &&
outputEmptyFlowFileOnZeroResults) {
+session.transfer((fileToProcess == null) ?
session.create() : session.create(fileToProcess), REL_SUCCESS);
--- End diff --
Good call, will make the change.
> Allow GenerateTableFetch to send empty flow files when no rows would be
> fetched
> ---
>
> Key: NIFI-5604
> URL: https://issues.apache.org/jira/browse/NIFI-5604
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>Priority: Major
>
> Currently, GenerateTableFetch will not output a flow file if there are no
> rows to be fetched. However, it may be desired (especially with incoming flow
> files) that a flow file be sent out even if GTF does not generate any SQL.
> This capability, along with the fragment attributes from NIFI-5601, would
> allow the user to handle this downstream.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5604) Allow GenerateTableFetch to send empty flow files when no rows would be fetched
[
https://issues.apache.org/jira/browse/NIFI-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685476#comment-16685476
]
ASF GitHub Bot commented on NIFI-5604:
--
Github user patricker commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3075#discussion_r233132498
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GenerateTableFetch.java
---
@@ -434,48 +448,53 @@ public void onTrigger(final ProcessContext context,
final ProcessSessionFactory
numberOfFetches = (partitionSize == 0) ? 1 : (rowCount
/ partitionSize) + (rowCount % partitionSize == 0 ? 0 : 1);
}
-// Generate SQL statements to read "pages" of data
-Long limit = partitionSize == 0 ? null : (long)
partitionSize;
-final String fragmentIdentifier =
UUID.randomUUID().toString();
-for (long i = 0; i < numberOfFetches; i++) {
-// Add a right bounding for the partitioning column if
necessary (only on last partition, meaning we don't need the limit)
-if ((i == numberOfFetches - 1) &&
useColumnValsForPaging && (maxValueClauses.isEmpty() || customWhereClause !=
null)) {
-maxValueClauses.add(columnForPartitioning + " <= "
+ maxValueForPartitioning);
-limit = null;
-}
+// If there are no SQL statements to be generated, still
output an empty flow file if specified by the user
+if (numberOfFetches == 0 &&
outputEmptyFlowFileOnZeroResults) {
+session.transfer((fileToProcess == null) ?
session.create() : session.create(fileToProcess), REL_SUCCESS);
--- End diff --
@mattyb149 It's not just the `fragment` attributes, I think you should add
all the standard attributes. Right now this Flowfile get's routed to success
with no attributes at all.
I would suggest refactoring this to include all the standard attributes
like `tablename`, `offset` etc... just like a real FlowFile would have. This
will make it much easier for downstream processors to use this file like
normal. I think the existing logic would cause `offset` to be `null`, which
would make sense for this scenario too.
> Allow GenerateTableFetch to send empty flow files when no rows would be
> fetched
> ---
>
> Key: NIFI-5604
> URL: https://issues.apache.org/jira/browse/NIFI-5604
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>Priority: Major
>
> Currently, GenerateTableFetch will not output a flow file if there are no
> rows to be fetched. However, it may be desired (especially with incoming flow
> files) that a flow file be sent out even if GTF does not generate any SQL.
> This capability, along with the fragment attributes from NIFI-5601, would
> allow the user to handle this downstream.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5604) Allow GenerateTableFetch to send empty flow files when no rows would be fetched
[ https://issues.apache.org/jira/browse/NIFI-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685470#comment-16685470 ] ASF GitHub Bot commented on NIFI-5604: -- Github user patricker commented on the issue: https://github.com/apache/nifi/pull/3075 Reviewing > Allow GenerateTableFetch to send empty flow files when no rows would be > fetched > --- > > Key: NIFI-5604 > URL: https://issues.apache.org/jira/browse/NIFI-5604 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > > Currently, GenerateTableFetch will not output a flow file if there are no > rows to be fetched. However, it may be desired (especially with incoming flow > files) that a flow file be sent out even if GTF does not generate any SQL. > This capability, along with the fragment attributes from NIFI-5601, would > allow the user to handle this downstream. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-648) add processor and add processor with linkage nomenclature is confusing
[ https://issues.apache.org/jira/browse/MINIFICPP-648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685438#comment-16685438 ] ASF GitHub Bot commented on MINIFICPP-648: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/432 > add processor and add processor with linkage nomenclature is confusing > -- > > Key: MINIFICPP-648 > URL: https://issues.apache.org/jira/browse/MINIFICPP-648 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Mr TheSegfault >Assignee: Arpad Boda >Priority: Blocker > Labels: CAPI > Fix For: 0.6.0 > > > add_processor should be changed to always add a processor with linkage > without compelling documentation as why this exists.. As a result we will > need to add a create_processor function to create one without adding it to > the flow ( certain use cases where a flow isn't needed such as invokehttp or > listenhttp ) this can be moved to 0.7.0 if we tag before recent commits. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4130) TransformXml - provide a way to define XSLT without external files
[
https://issues.apache.org/jira/browse/NIFI-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685334#comment-16685334
]
ASF GitHub Bot commented on NIFI-4130:
--
Github user bdesert commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1953#discussion_r233019686
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/TransformXml.java
---
@@ -82,12 +94,32 @@
public static final PropertyDescriptor XSLT_FILE_NAME = new
PropertyDescriptor.Builder()
.name("XSLT file name")
-.description("Provides the name (including full path) of the
XSLT file to apply to the flowfile XML content.")
-.required(true)
+.description("Provides the name (including full path) of the
XSLT file to apply to the flowfile XML content."
++ "One of the XSLT file name and XSLT controller
properties must be defined.")
+.required(false)
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
.build();
+public static final PropertyDescriptor XSLT_CONTROLLER = new
PropertyDescriptor.Builder()
+.name("xslt-controller")
+.displayName("XSLT controller")
+.description("Controller used to store XSLT definitions. One
of the XSLT file name and "
++ "XSLT controller properties must be defined.")
+.required(false)
+.identifiesControllerService(StringLookupService.class)
+.build();
+
+public static final PropertyDescriptor XSLT_CONTROLLER_KEY = new
PropertyDescriptor.Builder()
+.name("xslt-controller-key")
+.displayName("XSLT controller key")
+.description("Key used to retrieve the XSLT definition from
the XSLT controller. This property must be set when using "
++ "the XSLT controller property.")
+.required(false)
+
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
--- End diff --
since this property supports EL, shouldn't it be NON_EMPTY_**EL**_VALIDATOR?
> TransformXml - provide a way to define XSLT without external files
> --
>
> Key: NIFI-4130
> URL: https://issues.apache.org/jira/browse/NIFI-4130
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Major
>
> In cluster deployments the need to reference external configuration files can
> be annoying since it requires to access to all the NiFi nodes and to
> correctly deploy the files. It would be interesting to leverage the lookup
> controller services in TransformXml to provide a way to define XSLT directly
> from the UI without external configuration files.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4130) TransformXml - provide a way to define XSLT without external files
[
https://issues.apache.org/jira/browse/NIFI-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685335#comment-16685335
]
ASF GitHub Bot commented on NIFI-4130:
--
Github user bdesert commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1953#discussion_r233016908
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/TransformXml.java
---
@@ -82,12 +94,32 @@
public static final PropertyDescriptor XSLT_FILE_NAME = new
PropertyDescriptor.Builder()
.name("XSLT file name")
-.description("Provides the name (including full path) of the
XSLT file to apply to the flowfile XML content.")
-.required(true)
+.description("Provides the name (including full path) of the
XSLT file to apply to the flowfile XML content."
++ "One of the XSLT file name and XSLT controller
properties must be defined.")
+.required(false)
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
.build();
+public static final PropertyDescriptor XSLT_CONTROLLER = new
PropertyDescriptor.Builder()
+.name("xslt-controller")
+.displayName("XSLT controller")
--- End diff --
"XSLT Lookup" Would be more readable.
Description: "Lookup controller used to store..."
And: XSLT_CONTROLLER_KEY: "XSLT Lookup Key" (description looks fine)
> TransformXml - provide a way to define XSLT without external files
> --
>
> Key: NIFI-4130
> URL: https://issues.apache.org/jira/browse/NIFI-4130
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Major
>
> In cluster deployments the need to reference external configuration files can
> be annoying since it requires to access to all the NiFi nodes and to
> correctly deploy the files. It would be interesting to leverage the lookup
> controller services in TransformXml to provide a way to define XSLT directly
> from the UI without external configuration files.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5333) Create GetMongoRecord processor
[ https://issues.apache.org/jira/browse/NIFI-5333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685330#comment-16685330 ] ASF GitHub Bot commented on NIFI-5333: -- Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3011 Thanks @zenfenan. Could you and/or @mattyb149 take a look at 4975 as well sometime? That's the GridFS support. Would really like to give NiFi the ability to work natively with that. > Create GetMongoRecord processor > --- > > Key: NIFI-5333 > URL: https://issues.apache.org/jira/browse/NIFI-5333 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > > A processor similar to GetMongo that uses the record API should be created. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5791) Add Apache Daffodil parse/unparse processor
[ https://issues.apache.org/jira/browse/NIFI-5791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685210#comment-16685210 ] ASF GitHub Bot commented on NIFI-5791: -- Github user stevedlawrence commented on the issue: https://github.com/apache/nifi/pull/3130 After some research and reading the [Avro specification](https://avro.apache.org/docs/1.8.1/spec.html) , I'd agree that the DFDL infoset does seem somewhat similar to a Record. DFDL does support all the primitive types (null, boolean, int, long, float, double, bytes, string) and logical types (date, time, decimal), plus a few others (integer, byte, short, signed/unsigned) . But as far as complex types, it only really supports "records" and arrays. Below is the list of things in Avro that the DFDL infoset does not support: * It sort of supports enums, but only in the sense that it can validate that a primitive type is one of the valid enum values via the xsd:restriction. * Maps. In DFDL, a map would be implemented as a sequence of key/value pairs, so there wouldn't be any enforcement of unique keys. * Unions. Each element in the infoset must have an explicit primitive type. Each element can be optional or nulled, but cannot be a union of multiple primitive types. In DFDL, that is handled by an xs:choice of two different elements with different types and some method (often a discriminator) to determine which branch of the choice. * Namespaces are slightly different, but probably similar enough. DFDL uses XML namespacing. * Aliases are not supported. * Sort order. DFDL outputs infoset elements in the order in which they appear in schema. * The DFDL infoset does not contain the schema. One must keep track of the associated schema outside of the data. * The isn't really a concept of different serializations like Avro looks to have. Instead, the DFDL schema defines the physical data format via DFDL annotations, which are used to determine how to serialize/deserialize data. Theoretically, one could have different schemas with the same logical format but with with different DFDL annotations to describe physical formats, but that isn't a comment use case we've come across. The Daffodil devs would be happen to discuss the possibility of integrating Daffodil/DFDL as an alternative to Avro. Not sure if any of the above limitations are blockers. The core of Avro and DFDL definitely do seem to have some overlap. > Add Apache Daffodil parse/unparse processor > --- > > Key: NIFI-5791 > URL: https://issues.apache.org/jira/browse/NIFI-5791 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Steve Lawrence >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFIREG-188) Login by hitting Enter in the password field in web UI
[ https://issues.apache.org/jira/browse/NIFIREG-188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685163#comment-16685163 ] ASF GitHub Bot commented on NIFIREG-188: GitHub user kotarot opened a pull request: https://github.com/apache/nifi-registry/pull/147 NIFIREG-188: Login by hitting Enter in the input fields in web UI In this improvement, I appended the angular `keyup` event on the both input fields to achieve the feature. Could anyone review it please? You can merge this pull request into a Git repository by running: $ git pull https://github.com/kotarot/nifi-registry NIFIREG-188 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-registry/pull/147.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #147 commit f72f9dd8e0fafdb929b479041ad16833e378b0e9 Author: Kotaro Terada Date: 2018-11-13T12:16:37Z NIFIREG-188: Login by hitting Enter in the input fields in web UI NIFIREG-188: Fix the case 'keyUp' -> 'keyup' > Login by hitting Enter in the password field in web UI > -- > > Key: NIFIREG-188 > URL: https://issues.apache.org/jira/browse/NIFIREG-188 > Project: NiFi Registry > Issue Type: Improvement >Affects Versions: 0.1.0, 0.2.0 >Reporter: Julian Gimbel >Priority: Minor > > While logging in to Nifi Registry it should be possible to hit enter in the > password field to login instead of using the mouse or tab on to the login > button. > Unfortunately I am not experienced enough with angular to do it myself. > It is not as simple as just adding a tag around the dialog fields and > buttons. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
