[jira] [Created] (NIFI-7639) JMSPublisher incorrectly uses JMS_Expiration and causes NumberFormatExpection
Darren Hitchman created NIFI-7639:
-
Summary: JMSPublisher incorrectly uses JMS_Expiration and causes
NumberFormatExpection
Key: NIFI-7639
URL: https://issues.apache.org/jira/browse/NIFI-7639
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Affects Versions: 1.11.4
Reporter: Darren Hitchman
Assignee: Darren Hitchman
{color:#80}JMSPublisher incorrectly sets the{color}{color:#80}
TimeToLive when it should set the expiry of the message.{color}
{color:#80}This becomes an issue when using ConsumeJMS as it sets the
jms_expiry to the expected value (ms timestamp) despite PublishJMS not being
compatible (expecting ms duration for ttl). {color}
{color:#80}JMS expiry is defined as the timestamp (ms) when the messages
expires where as the setTimeToLive defined in the jmsTemplate is referring to
the number of ms from current time until expiry,{color}
{color:#80}else if
{color}(entry.getKey().equals(JmsHeaders.{color:#660e7a}EXPIRATION{color})) {
{color:#80}this{color}.{color:#660e7a}jmsTemplate{color}.setTimeToLive(Integer.parseInt(entry.getValue()));
{color:#80}this{color}.{color:#660e7a}jmsTemplate{color}.setExplicitQosEnabled({color:#80}true{color});
}
*Integer.parseInt fails to parse the Long that is defined in the attributes
jms_expiration.
A workout for this bug would be to use UpdateAttributes to change the
jms_expiration value from a timestamp to a duration value e.g.
ttl=jms_expiration-now() how ever PublishJMS should correctly honor the
intended value.
If we wish to continue using the spring framework for the TTL then a possible
fix would be
{color:#80}this{color}.{color:#660e7a}jmsTemplate{color}.setTimeToLive(
System.currentTimeMillis() - Long.parseLong(entry.getValue()));
however I think we need to not break logic based on existing presumptions by
users. We can for values less then 214748364.7 seconds (about 7 years) as per
[https://www.ibm.com/support/knowledgecenter/SSFKSJ_7.5.0/com.ibm.mq.dev.doc/q032030_.htm]
. Any values less than this is presumed to be TTL rather than expiry
more info here.
http://activemq.apache.org/components/artemis/documentation/1.0.0/message-expiry.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] mattyb149 commented on pull request #2231: NIFI-4521 MS SQL CDC Processor
mattyb149 commented on pull request #2231: URL: https://github.com/apache/nifi/pull/2231#issuecomment-658511456 @patricker Can you change this PR to be based against the `main` branch and rebase against said branch? Please and thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on pull request #2231: NIFI-4521 MS SQL CDC Processor
mattyb149 commented on pull request #2231: URL: https://github.com/apache/nifi/pull/2231#issuecomment-658511293 @qingweihuang It looks like you are building from the nifi-nar-bundles directory, but you'll want to run the `mvn` command from the top level in order to build all the dependencies, common modules, etc. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] qingweihuang commented on pull request #2231: NIFI-4521 MS SQL CDC Processor
qingweihuang commented on pull request #2231: URL: https://github.com/apache/nifi/pull/2231#issuecomment-658503782 I execute "mvn clean install" and the build fails. What are the possible reasons? Thank you! `[INFO] nifi-nar-bundles ... SUCCESS [ 3.222 s] [INFO] nifi-cdc ... SUCCESS [ 0.114 s] [INFO] nifi-cdc-api ... SUCCESS [ 9.681 s] [INFO] nifi-cdc-mssql-bundle .. SUCCESS [ 0.097 s] [INFO] nifi-cdc-mssql-processors .. FAILURE [ 13.863 s] [INFO] nifi-cdc-mssql-nar . SKIPPED [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 30.073 s [INFO] Finished at: 2020-07-15T10:05:46+08:00 [INFO] [ERROR] Failed to execute goal on project nifi-cdc-mssql-processors: Could not resolve dependencies for project org.apache.nifi:nifi-cdc-mssql-processors:jar:1.12.0-SNAPSHOT: The following artifacts could not be resolved: org.apache.nifi:nifi-mock:jar:1.12.0-SNAPSHOT, org.apache.nifi:nifi-dbcp-service:jar:1.12.0-SNAPSHOT, org.apache.nifi:nifi-mock-record-utils:jar:1.12.0-SNAPSHOT: Could not find artifact org.apache.nifi:nifi-mock:jar:1.12.0-SNAPSHOT in apache.snapshots (https://repository.apache.org/snapshots) -> [Help 1]` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] qingweihuang removed a comment on pull request #2231: NIFI-4521 MS SQL CDC Processor
qingweihuang removed a comment on pull request #2231: URL: https://github.com/apache/nifi/pull/2231#issuecomment-658503475 I execute "mvn clean install" and the build fails. What are the possible reasons? Thank you! [INFO] Reactor Summary for nifi-nar-bundles 1.12.0-SNAPSHOT: [INFO] [INFO] nifi-nar-bundles ... SUCCESS [ 3.222 s] [INFO] nifi-cdc ... SUCCESS [ 0.114 s] [INFO] nifi-cdc-api ... SUCCESS [ 9.681 s] [INFO] nifi-cdc-mssql-bundle .. SUCCESS [ 0.097 s] [INFO] nifi-cdc-mssql-processors .. FAILURE [ 13.863 s] [INFO] nifi-cdc-mssql-nar . SKIPPED [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 30.073 s [INFO] Finished at: 2020-07-15T10:05:46+08:00 [INFO] [ERROR] Failed to execute goal on project nifi-cdc-mssql-processors: Could not resolve dependencies for project org.apache.nifi:nifi-cdc-mssql-processors:jar:1.12.0-SNAPSHOT: The following artifacts could not be resolved: org.apache.nifi:nifi-mock:jar:1.12.0-SNAPSHOT, org.apache.nifi:nifi-dbcp-service:jar:1.12.0-SNAPSHOT, org.apache.nifi:nifi-mock-record-utils:jar:1.12.0-SNAPSHOT: Could not find artifact org.apache.nifi:nifi-mock:jar:1.12.0-SNAPSHOT in apache.snapshots (https://repository.apache.org/snapshots) -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :nifi-cdc-mssql-processors This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] qingweihuang commented on pull request #2231: NIFI-4521 MS SQL CDC Processor
qingweihuang commented on pull request #2231: URL: https://github.com/apache/nifi/pull/2231#issuecomment-658503475 I execute "mvn clean install" and the build fails. What are the possible reasons? Thank you! [INFO] Reactor Summary for nifi-nar-bundles 1.12.0-SNAPSHOT: [INFO] [INFO] nifi-nar-bundles ... SUCCESS [ 3.222 s] [INFO] nifi-cdc ... SUCCESS [ 0.114 s] [INFO] nifi-cdc-api ... SUCCESS [ 9.681 s] [INFO] nifi-cdc-mssql-bundle .. SUCCESS [ 0.097 s] [INFO] nifi-cdc-mssql-processors .. FAILURE [ 13.863 s] [INFO] nifi-cdc-mssql-nar . SKIPPED [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 30.073 s [INFO] Finished at: 2020-07-15T10:05:46+08:00 [INFO] [ERROR] Failed to execute goal on project nifi-cdc-mssql-processors: Could not resolve dependencies for project org.apache.nifi:nifi-cdc-mssql-processors:jar:1.12.0-SNAPSHOT: The following artifacts could not be resolved: org.apache.nifi:nifi-mock:jar:1.12.0-SNAPSHOT, org.apache.nifi:nifi-dbcp-service:jar:1.12.0-SNAPSHOT, org.apache.nifi:nifi-mock-record-utils:jar:1.12.0-SNAPSHOT: Could not find artifact org.apache.nifi:nifi-mock:jar:1.12.0-SNAPSHOT in apache.snapshots (https://repository.apache.org/snapshots) -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :nifi-cdc-mssql-processors This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-7638) Add PBE AEAD sensitive flow property protection scheme
Andy LoPresto created NIFI-7638:
---
Summary: Add PBE AEAD sensitive flow property protection scheme
Key: NIFI-7638
URL: https://issues.apache.org/jira/browse/NIFI-7638
Project: Apache NiFi
Issue Type: Improvement
Components: Configuration Management, Core Framework
Affects Versions: 1.11.4
Reporter: Andy LoPresto
Assignee: Andy LoPresto
A user requested a change from AES-CBC to AES-G/CM for the
{{nifi.sensitive.props.algorithm}} in {{nifi.properties}}. The current possible
values are all {{EncryptionMethod}} enum values, which includes raw
(directly-keyed vs. PBE) AES-G/CM, but this would require a valid
hexadecimal-encoded AES key in the {{nifi.sensitive.props.key}} value. One or
more new {{EncryptionMethod}} entries which combine reasonable default values
for a KDF (Argon2, bcrypt, scrypt, PBKDF2) and AEAD mode of operation
(AES-G/CM) would allow for simpler configuration and migration. The other
option is to enhance the {{EncryptionMethod}} enum values with custom values in
the {{NiFiProperties}} or {{StringEncryptor}} class which provide an additional
level of security without modifying the {{EncryptionMethod}} enum directly, as
the {{EncryptContent}} processor already allows independent configuration of a
KDF and cipher algorithm (see NIFI-7122 / [PR
4228|https://github.com/apache/nifi/pull/4228]).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-7636) ValidateRecord Prints Array Paths in Confusing Way
[
https://issues.apache.org/jira/browse/NIFI-7636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Secules updated NIFI-7636:
---
Summary: ValidateRecord Prints Array Paths in Confusing Way (was: Validate
Record Prints Array Paths in Confusing Way)
> ValidateRecord Prints Array Paths in Confusing Way
> --
>
> Key: NIFI-7636
> URL: https://issues.apache.org/jira/browse/NIFI-7636
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.11.4
>Reporter: Eric Secules
>Priority: Major
>
> When there is a avro Validation Error in an element of a JSON array the array
> index is placed before the array's name in the path. For example if the input
> JSON was this:
> {code:java}
> {
> "myArray": [
> {
> "myValue": null
> }
> ]
> }
> {code}
> and "myValue" was required
> the validation error would be:
>
> {code:java}
> {
> "fieldName" : "[0]/myArray/myValue",
> "inputValue" : null,
> "explanation" : "Field is required",
> "type" : "MISSING_FIELD"
> }
> {code}
> It would make more sense if the array index was after the array field name
> like so. Placing the array index before the array doesn't make any sense to
> me.
> {code:java}
> {
> "fieldName" : "/myArray[0]/myValue",
> "inputValue" : null,
> "explanation" : "Field is required",
> "type" : "MISSING_FIELD"
> }
> {code}
> or if the field name was made using JSONPath syntax like:
> {code:java}
> {
> "fieldName" : "$.myArray[0].myValue",
> "inputValue" : null,
> "explanation" : "Field is required",
> "type" : "MISSING_FIELD"
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (NIFI-7637) ValidateRecord does not validate primative datatypes in an array
Eric Secules created NIFI-7637:
--
Summary: ValidateRecord does not validate primative datatypes in
an array
Key: NIFI-7637
URL: https://issues.apache.org/jira/browse/NIFI-7637
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 1.11.4
Reporter: Eric Secules
Given this input record
{code:java}
{
"numbers":[{"foo":1},"2",3,"ASD"]
}
{code}
and this Avro schema
{code:java}
{
"name": "MyClass",
"type": "record",
"namespace": "com.acme.avro",
"fields": [
{
"name": "numbers",
"type": {
"type": "array",
"items": "int"
}
}
]
}
{code}
*Expected Results*
I would expect the record to fail validation even with strict type checking
turned off. Because the content of the array cannot all be treated as an
integer.
*Actual Results*
The record gets routed a flowfile on the success relationship.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #807: MINIFICPP-1228 - Resource ownership refactor + flowFile-owning processors.
szaszm commented on a change in pull request #807:
URL: https://github.com/apache/nifi-minifi-cpp/pull/807#discussion_r453579663
##
File path: libminifi/include/core/Repository.h
##
@@ -228,6 +232,8 @@ class Repository : public virtual
core::SerializableComponent, public core::Trac
Repository =(const Repository ) = delete;
protected:
+ std::map> containers;
Review comment:
This could use a comment, "containers" is not clear IMO. What can be
containers, and what not? Does it include flow files, connections, processors,
S2S client, etc.? All of these are derived from `Connectable`.
##
File path: libminifi/test/persistence-tests/PersistenceTests.cpp
##
@@ -0,0 +1,218 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+#include
+#include
+#include
+#include
+
+#include "core/Core.h"
+#include "core/repository/AtomicRepoEntries.h"
+#include "core/RepositoryFactory.h"
+#include "FlowFileRecord.h"
+#include "FlowFileRepository.h"
+#include "properties/Configure.h"
+#include "../unit/ProvenanceTestHelper.h"
+#include "../TestBase.h"
+#include "../../extensions/libarchive/MergeContent.h"
+#include "../test/BufferReader.h"
+
+using Connection = minifi::Connection;
+using MergeContent = minifi::processors::MergeContent;
+
+struct TestFlow{
+ TestFlow(const std::shared_ptr&
ff_repository, const std::shared_ptr& content_repo,
const std::shared_ptr& prov_repo)
+ : ff_repository(ff_repository), content_repo(content_repo),
prov_repo(prov_repo) {
+std::shared_ptr
controller_services_provider = nullptr;
+
+// setup MERGE processor
+{
+ merge = std::make_shared("MergeContent", mergeProcUUID());
+ merge->initialize();
+ merge->setAutoTerminatedRelationships({{"original", "d"}});
+
+ merge->setProperty(MergeContent::MergeFormat, MERGE_FORMAT_CONCAT_VALUE);
+ merge->setProperty(MergeContent::MergeStrategy, MERGE_STRATEGY_BIN_PACK);
+ merge->setProperty(MergeContent::DelimiterStrategy,
DELIMITER_STRATEGY_TEXT);
+ merge->setProperty(MergeContent::MinEntries, "3");
+ merge->setProperty(MergeContent::Header, "_Header_");
+ merge->setProperty(MergeContent::Footer, "_Footer_");
+ merge->setProperty(MergeContent::Demarcator, "_Demarcator_");
+ merge->setProperty(MergeContent::MaxBinAge, "1 h");
+
+ std::shared_ptr node =
std::make_shared(merge);
+ mergeContext = std::make_shared(node,
controller_services_provider, prov_repo, ff_repository, content_repo);
+}
+
+// setup INPUT processor
+{
+ inputProcessor = std::make_shared("source",
inputProcUUID());
+ std::shared_ptr node =
std::make_shared(inputProcessor);
+ inputContext = std::make_shared(node,
controller_services_provider, prov_repo,
+ff_repository,
content_repo);
+}
+
+// setup Input Connection
+{
+ input = std::make_shared(ff_repository, content_repo,
"Input", inputConnUUID());
+ input->setRelationship({"input", "d"});
+ input->setDestinationUUID(mergeProcUUID());
+ input->setSourceUUID(inputProcUUID());
+ inputProcessor->addConnection(input);
+}
+
+// setup Output Connection
+{
+ output = std::make_shared(ff_repository, content_repo,
"Output", outputConnUUID());
+ output->setRelationship(MergeContent::Merge);
+ output->setSourceUUID(mergeProcUUID());
+}
+
+// setup ProcessGroup
+{
+ root =
std::make_shared(core::ProcessGroupType::ROOT_PROCESS_GROUP,
"root");
+ root->addProcessor(merge);
+ root->addConnection(input);
+ root->addConnection(output);
+}
+
+// prepare Merge Processor for execution
+merge->setScheduledState(core::ScheduledState::RUNNING);
+merge->onSchedule(mergeContext.get(), new
core::ProcessSessionFactory(mergeContext));
+ }
+ void write(const std::string& data) {
+minifi::io::DataStream stream(reinterpret_cast(data.c_str()), data.length());
+core::ProcessSession sessionGenFlowFile(inputContext);
+std::shared_ptr flow =
std::static_pointer_cast(sessionGenFlowFile.create());
+sessionGenFlowFile.importFrom(stream, flow);
[jira] [Updated] (NIFI-7636) Validate Record Prints Array Paths in Confusing Way
[
https://issues.apache.org/jira/browse/NIFI-7636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Secules updated NIFI-7636:
---
Description:
When there is a avro Validation Error in an element of a JSON array the array
index is placed before the array's name in the path. For example if the input
JSON was this:
{code:java}
{
"myArray": [
{
"myValue": null
}
]
}
{code}
and "myValue" was required
the validation error would be:
{code:java}
{
"fieldName" : "[0]/myArray/myValue",
"inputValue" : null,
"explanation" : "Field is required",
"type" : "MISSING_FIELD"
}
{code}
It would make more sense if the array index was after the array field name like
so. Placing the array index before the array doesn't make any sense to me.
{code:java}
{
"fieldName" : "/myArray[0]/myValue",
"inputValue" : null,
"explanation" : "Field is required",
"type" : "MISSING_FIELD"
}
{code}
or if the field name was made using JSONPath syntax like:
{code:java}
{
"fieldName" : "$.myArray[0].myValue",
"inputValue" : null,
"explanation" : "Field is required",
"type" : "MISSING_FIELD"
}
{code}
was:
When there is a avro Validation Error in an element of a JSON array the array
index is placed before the array's name in the path. For example if the input
JSON was this:
{code:java}
{
"myArray": [
{
"myValue": null
}
]
}
{code}
and "myValue" was required
the validation error would be:
{code:java}
{
"fieldName" : "[0]/myArray/myValue",
"inputValue" : null,
"explanation" : "Field is required",
"type" : "MISSING_FIELD"
}
{code}
It would make more sense if the array index was after the array field name like
so. Placing the array index before the array doesn't make any sense to me.
{code:java}
{
"fieldName" : "/myArray/[0]/myValue",
"inputValue" : null,
"explanation" : "Field is required",
"type" : "MISSING_FIELD"
}
{code}
or if the field name was made using JSONPath syntax like:
{code:java}
{
"fieldName" : "$.myArray[0].myValue",
"inputValue" : null,
"explanation" : "Field is required",
"type" : "MISSING_FIELD"
}
{code}
> Validate Record Prints Array Paths in Confusing Way
> ---
>
> Key: NIFI-7636
> URL: https://issues.apache.org/jira/browse/NIFI-7636
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.11.4
>Reporter: Eric Secules
>Priority: Major
>
> When there is a avro Validation Error in an element of a JSON array the array
> index is placed before the array's name in the path. For example if the input
> JSON was this:
> {code:java}
> {
> "myArray": [
> {
> "myValue": null
> }
> ]
> }
> {code}
> and "myValue" was required
> the validation error would be:
>
> {code:java}
> {
> "fieldName" : "[0]/myArray/myValue",
> "inputValue" : null,
> "explanation" : "Field is required",
> "type" : "MISSING_FIELD"
> }
> {code}
> It would make more sense if the array index was after the array field name
> like so. Placing the array index before the array doesn't make any sense to
> me.
> {code:java}
> {
> "fieldName" : "/myArray[0]/myValue",
> "inputValue" : null,
> "explanation" : "Field is required",
> "type" : "MISSING_FIELD"
> }
> {code}
> or if the field name was made using JSONPath syntax like:
> {code:java}
> {
> "fieldName" : "$.myArray[0].myValue",
> "inputValue" : null,
> "explanation" : "Field is required",
> "type" : "MISSING_FIELD"
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-7304) Default value for content length filter blocks Site to Site communication
[
https://issues.apache.org/jira/browse/NIFI-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157671#comment-17157671
]
ASF subversion and git services commented on NIFI-7304:
---
Commit 77078a85d921070c24903b6e79aa20e0493a7f9b in nifi's branch
refs/heads/main from Joe Witt
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=77078a8 ]
NIFI-7304 resolving a contrib check issue with line length
> Default value for content length filter blocks Site to Site communication
> -
>
> Key: NIFI-7304
> URL: https://issues.apache.org/jira/browse/NIFI-7304
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.12.0
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Blocker
> Labels: cluster, filter, http, replication, security,
> site-to-site
> Fix For: 1.12.0
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> When the content-length filter was introduced in NIFI-7153, it did not
> separate Site to Site (S2S) or cluster request replication requests from
> user-generated requests. With the default value of 20 MB, it is very likely
> that legitimate requests of this nature will be unexpectedly blocked.
> The immediate fix is to change the default value in {{nifi.properties}} to
> empty and only enable this functionality when a value is provided.
> A subtask will be opened to investigate if these requests should be excluded
> from the length limiting filter (either by convention or via an
> admin-enumerated exclusion list/setting).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-7636) Validate Record Prints Array Paths in Confusing Way
[
https://issues.apache.org/jira/browse/NIFI-7636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Secules updated NIFI-7636:
---
Description:
When there is a avro Validation Error in an element of a JSON array the array
index is placed before the array's name in the path. For example if the input
JSON was this:
{code:java}
{
"myArray": [
{
"myValue": null
}
]
}
{code}
and "myValue" was required
the validation error would be:
{code:java}
{
"fieldName" : "[0]/myArray/myValue",
"inputValue" : null,
"explanation" : "Field is required",
"type" : "MISSING_FIELD"
}
{code}
It would make more sense if the array index was after the array field name like
so. Placing the array index before the array doesn't make any sense to me.
{code:java}
{
"fieldName" : "/myArray/[0]/myValue",
"inputValue" : null,
"explanation" : "Field is required",
"type" : "MISSING_FIELD"
}
{code}
or if the field name was made using JSONPath syntax like:
{code:java}
{
"fieldName" : "$.myArray[0].myValue",
"inputValue" : null,
"explanation" : "Field is required",
"type" : "MISSING_FIELD"
}
{code}
was:
When there is a avro Validation Error in an element of a JSON array the array
index is placed before the array's name in the path. For example if the input
JSON was this:
{code:java}
{
"myArray": [
{
"myValue": null
}
]
}
{code}
and "myValue" was required
the validation error would be:
{code:java}
{
"fieldName" : "[0]/myArray/myValue",
"inputValue" : null,
"explanation" : "Field is required",
"type" : "MISSING_FIELD"
}
{code}
It would make more sense if the array index was after the array field name like
so. Placing the array index before the array doesn't make any sense to me.
{code:java}
{
"fieldName" : "/myArray/[0]/myValue",
"inputValue" : null,
"explanation" : "Field is required",
"type" : "MISSING_FIELD"
}
{code}
or if the field name was made using JSONPath syntax like $.myArray[0].myValue.
> Validate Record Prints Array Paths in Confusing Way
> ---
>
> Key: NIFI-7636
> URL: https://issues.apache.org/jira/browse/NIFI-7636
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.11.4
>Reporter: Eric Secules
>Priority: Major
>
> When there is a avro Validation Error in an element of a JSON array the array
> index is placed before the array's name in the path. For example if the input
> JSON was this:
> {code:java}
> {
> "myArray": [
> {
> "myValue": null
> }
> ]
> }
> {code}
> and "myValue" was required
> the validation error would be:
>
> {code:java}
> {
> "fieldName" : "[0]/myArray/myValue",
> "inputValue" : null,
> "explanation" : "Field is required",
> "type" : "MISSING_FIELD"
> }
> {code}
> It would make more sense if the array index was after the array field name
> like so. Placing the array index before the array doesn't make any sense to
> me.
> {code:java}
> {
> "fieldName" : "/myArray/[0]/myValue",
> "inputValue" : null,
> "explanation" : "Field is required",
> "type" : "MISSING_FIELD"
> }
> {code}
> or if the field name was made using JSONPath syntax like:
> {code:java}
> {
> "fieldName" : "$.myArray[0].myValue",
> "inputValue" : null,
> "explanation" : "Field is required",
> "type" : "MISSING_FIELD"
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (NIFI-7636) Validate Record Prints Array Paths in Confusing Way
Eric Secules created NIFI-7636:
--
Summary: Validate Record Prints Array Paths in Confusing Way
Key: NIFI-7636
URL: https://issues.apache.org/jira/browse/NIFI-7636
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 1.11.4
Reporter: Eric Secules
When there is a avro Validation Error in an element of a JSON array the array
index is placed before the array's name in the path. For example if the input
JSON was this:
{code:java}
{
"myArray": [
{
"myValue": null
}
]
}
{code}
and "myValue" was required
the validation error would be:
{code:java}
{
"fieldName" : "[0]/myArray/myValue",
"inputValue" : null,
"explanation" : "Field is required",
"type" : "MISSING_FIELD"
}
{code}
It would make more sense if the array index was after the array field name like
so. Placing the array index before the array doesn't make any sense to me.
{code:java}
{
"fieldName" : "/myArray/[0]/myValue",
"inputValue" : null,
"explanation" : "Field is required",
"type" : "MISSING_FIELD"
}
{code}
or if the field name was made using JSONPath syntax like $.myArray[0].myValue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-7627) When updating Parameter Context, some processors may fail to restart
[ https://issues.apache.org/jira/browse/NIFI-7627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157632#comment-17157632 ] ASF subversion and git services commented on NIFI-7627: --- Commit f8a730568fcb7734fb0e70755581622d8261d424 in nifi's branch refs/heads/main from markap14 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=f8a7305 ] NIFI-7627: Fixed typo that showed the wrong case for Run Statuses in the ProcessorRunStatusDetailsDTO (#4400) > When updating Parameter Context, some processors may fail to restart > > > Key: NIFI-7627 > URL: https://issues.apache.org/jira/browse/NIFI-7627 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.12.0 >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Blocker > Fix For: 1.12.0 > > Time Spent: 20m > Remaining Estimate: 0h > > When a Parameter Context is updated, it must stop any processors that will be > affected by the update. The context is then updated, and any processor that > was stopped is then restarted (assuming that it is still valid after the > update). > In NIFI-7536 we improved the efficiency of waiting for processors to complete > the action of stopping. However, it appears that in this fix, we caused a new > problem. If a Processor that is affected by the Parameter Context is stopped, > and then takes a long time to complete validation, that Processor may not be > started again. > Note that this issue only affects the 'main' branch and hasn't yet been > released. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7627) When updating Parameter Context, some processors may fail to restart
[ https://issues.apache.org/jira/browse/NIFI-7627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende updated NIFI-7627: -- Resolution: Fixed Status: Resolved (was: Patch Available) > When updating Parameter Context, some processors may fail to restart > > > Key: NIFI-7627 > URL: https://issues.apache.org/jira/browse/NIFI-7627 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.12.0 >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Blocker > Fix For: 1.12.0 > > Time Spent: 20m > Remaining Estimate: 0h > > When a Parameter Context is updated, it must stop any processors that will be > affected by the update. The context is then updated, and any processor that > was stopped is then restarted (assuming that it is still valid after the > update). > In NIFI-7536 we improved the efficiency of waiting for processors to complete > the action of stopping. However, it appears that in this fix, we caused a new > problem. If a Processor that is affected by the Parameter Context is stopped, > and then takes a long time to complete validation, that Processor may not be > started again. > Note that this issue only affects the 'main' branch and hasn't yet been > released. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] bbende merged pull request #4400: NIFI-7627: Fixed typo that showed the wrong case for Run Statuses in …
bbende merged pull request #4400: URL: https://github.com/apache/nifi/pull/4400 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-7616) PrometheusReportingTask: Wrong number of processor active threads reported ?
[
https://issues.apache.org/jira/browse/NIFI-7616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Charron updated NIFI-7616:
---
Attachment: (was: nifi-metrics.diff)
> PrometheusReportingTask: Wrong number of processor active threads reported ?
>
>
> Key: NIFI-7616
> URL: https://issues.apache.org/jira/browse/NIFI-7616
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Benjamin Charron
>Priority: Trivial
> Labels: Metrics, reporting_task
> Fix For: 1.12.0
>
> Attachments: nifi-metrics.diff
>
>
> The number of _processor_ active threads reported by prometheus
> ("nifi_amount_threads_active") seem off.
> I think "{{status.getActiveThreadCount()}}" should be
> "{{processorStatus.getActiveThreadCount()}}" at
> [PrometheusMetricsUtil.java:193|https://github.com/apache/nifi/blob/aa741cc5967f62c3c38c2a47e712b7faa6fe19ff/nifi-nar-bundles/nifi-extension-utils/nifi-prometheus-utils/src/main/java/org/apache/nifi/prometheus/util/PrometheusMetricsUtil.java#L193]
> and :194
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-7304) Default value for content length filter blocks Site to Site communication
[
https://issues.apache.org/jira/browse/NIFI-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy LoPresto updated NIFI-7304:
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Default value for content length filter blocks Site to Site communication
> -
>
> Key: NIFI-7304
> URL: https://issues.apache.org/jira/browse/NIFI-7304
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.12.0
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Blocker
> Labels: cluster, filter, http, replication, security,
> site-to-site
> Fix For: 1.12.0
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> When the content-length filter was introduced in NIFI-7153, it did not
> separate Site to Site (S2S) or cluster request replication requests from
> user-generated requests. With the default value of 20 MB, it is very likely
> that legitimate requests of this nature will be unexpectedly blocked.
> The immediate fix is to change the default value in {{nifi.properties}} to
> empty and only enable this functionality when a value is provided.
> A subtask will be opened to investigate if these requests should be excluded
> from the length limiting filter (either by convention or via an
> admin-enumerated exclusion list/setting).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-7614) Improve terminology around top-level key management for sensitive configuration values
[
https://issues.apache.org/jira/browse/NIFI-7614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy LoPresto updated NIFI-7614:
Status: Patch Available (was: Open)
> Improve terminology around top-level key management for sensitive
> configuration values
> --
>
> Key: NIFI-7614
> URL: https://issues.apache.org/jira/browse/NIFI-7614
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Configuration, Configuration Management, Core Framework
>Affects Versions: 1.11.4
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Major
> Labels: security, terminology
>
> The top-level key and associated handling stored in {{bootstrap.conf}} and
> used to encrypt the sensitive properties in {{nifi.properties}}, etc. should
> be renamed from "master".
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-7614) Improve terminology around top-level key management for sensitive configuration values
[
https://issues.apache.org/jira/browse/NIFI-7614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy LoPresto updated NIFI-7614:
Fix Version/s: 1.12.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Improve terminology around top-level key management for sensitive
> configuration values
> --
>
> Key: NIFI-7614
> URL: https://issues.apache.org/jira/browse/NIFI-7614
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Configuration, Configuration Management, Core Framework
>Affects Versions: 1.11.4
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Major
> Labels: security, terminology
> Fix For: 1.12.0
>
>
> The top-level key and associated handling stored in {{bootstrap.conf}} and
> used to encrypt the sensitive properties in {{nifi.properties}}, etc. should
> be renamed from "master".
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-7621) Remove legacy terminology around branch names from GitHub messaging and developer documentation
[
https://issues.apache.org/jira/browse/NIFI-7621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy LoPresto updated NIFI-7621:
Status: Patch Available (was: Open)
> Remove legacy terminology around branch names from GitHub messaging and
> developer documentation
> ---
>
> Key: NIFI-7621
> URL: https://issues.apache.org/jira/browse/NIFI-7621
> Project: Apache NiFi
> Issue Type: Task
> Components: Documentation Website, Tools and Build
>Affects Versions: 1.11.4
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Major
> Labels: terminology
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Change references to {{master}} branch to reflect new name {{main}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-7621) Remove legacy terminology around branch names from GitHub messaging and developer documentation
[
https://issues.apache.org/jira/browse/NIFI-7621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy LoPresto updated NIFI-7621:
Fix Version/s: 1.12.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Remove legacy terminology around branch names from GitHub messaging and
> developer documentation
> ---
>
> Key: NIFI-7621
> URL: https://issues.apache.org/jira/browse/NIFI-7621
> Project: Apache NiFi
> Issue Type: Task
> Components: Documentation Website, Tools and Build
>Affects Versions: 1.11.4
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Major
> Labels: terminology
> Fix For: 1.12.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Change references to {{master}} branch to reflect new name {{main}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] alopresto closed pull request #4396: NIFI-7621 & NIFI-7614 Updated terminology throughout codebase and doc…
alopresto closed pull request #4396: URL: https://github.com/apache/nifi/pull/4396 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] alopresto commented on pull request #4396: NIFI-7621 & NIFI-7614 Updated terminology throughout codebase and doc…
alopresto commented on pull request #4396: URL: https://github.com/apache/nifi/pull/4396#issuecomment-658318494 Forgot the closing keyword. This PR was successfully merged in commit 0fa8776f4d2305b9cd5d46525e45dc73678dac52. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] alopresto commented on pull request #4354: NIFI-7304 Removed default value for nifi.web.max.content.size.
alopresto commented on pull request #4354: URL: https://github.com/apache/nifi/pull/4354#issuecomment-658318205 Forgot the closing keywords in the commit. This PR was successfully merged in commit dbee774c5b95121b2d89621fc66f8a215c15ad7c. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] alopresto closed pull request #4354: NIFI-7304 Removed default value for nifi.web.max.content.size.
alopresto closed pull request #4354: URL: https://github.com/apache/nifi/pull/4354 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-7304) Default value for content length filter blocks Site to Site communication
[
https://issues.apache.org/jira/browse/NIFI-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157542#comment-17157542
]
ASF subversion and git services commented on NIFI-7304:
---
Commit dbee774c5b95121b2d89621fc66f8a215c15ad7c in nifi's branch
refs/heads/main from Andy LoPresto
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=dbee774 ]
NIFI-7304 Removed default value for nifi.web.max.content.size.
Added Bundle#toString() method.
Refactored implementation of filter addition logic.
Added logging.
Added unit tests to check for filter enablement.
Introduced content-length exception handling in StandardPublicPort.
Added filter bypass functionality for framework requests in ContentLengthFilter.
Updated property documentation in Admin Guide.
Renamed methods & added Javadoc to clarify purpose of filters in JettyServer.
Cleaned up conditional logic in StandardPublicPort.
Moved ContentLengthFilterTest to correct module.
Refactored unit tests for accuracy and clarity.
Fixed remaining merge conflict due to method renaming.
Signed-off-by: Joe Witt
> Default value for content length filter blocks Site to Site communication
> -
>
> Key: NIFI-7304
> URL: https://issues.apache.org/jira/browse/NIFI-7304
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.12.0
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Blocker
> Labels: cluster, filter, http, replication, security,
> site-to-site
> Fix For: 1.12.0
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> When the content-length filter was introduced in NIFI-7153, it did not
> separate Site to Site (S2S) or cluster request replication requests from
> user-generated requests. With the default value of 20 MB, it is very likely
> that legitimate requests of this nature will be unexpectedly blocked.
> The immediate fix is to change the default value in {{nifi.properties}} to
> empty and only enable this functionality when a value is provided.
> A subtask will be opened to investigate if these requests should be excluded
> from the length limiting filter (either by convention or via an
> admin-enumerated exclusion list/setting).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-7614) Improve terminology around top-level key management for sensitive configuration values
[
https://issues.apache.org/jira/browse/NIFI-7614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157538#comment-17157538
]
ASF subversion and git services commented on NIFI-7614:
---
Commit 0fa8776f4d2305b9cd5d46525e45dc73678dac52 in nifi's branch
refs/heads/main from Andy LoPresto
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=0fa8776 ]
NIFI-7621 & NIFI-7614 Updated terminology throughout codebase and documentation.
Fixed unit test failures introduced from static imports during refactoring.
Signed-off-by: Joe Witt
> Improve terminology around top-level key management for sensitive
> configuration values
> --
>
> Key: NIFI-7614
> URL: https://issues.apache.org/jira/browse/NIFI-7614
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Configuration, Configuration Management, Core Framework
>Affects Versions: 1.11.4
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Major
> Labels: security, terminology
>
> The top-level key and associated handling stored in {{bootstrap.conf}} and
> used to encrypt the sensitive properties in {{nifi.properties}}, etc. should
> be renamed from "master".
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-7621) Remove legacy terminology around branch names from GitHub messaging and developer documentation
[
https://issues.apache.org/jira/browse/NIFI-7621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157537#comment-17157537
]
ASF subversion and git services commented on NIFI-7621:
---
Commit 0fa8776f4d2305b9cd5d46525e45dc73678dac52 in nifi's branch
refs/heads/main from Andy LoPresto
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=0fa8776 ]
NIFI-7621 & NIFI-7614 Updated terminology throughout codebase and documentation.
Fixed unit test failures introduced from static imports during refactoring.
Signed-off-by: Joe Witt
> Remove legacy terminology around branch names from GitHub messaging and
> developer documentation
> ---
>
> Key: NIFI-7621
> URL: https://issues.apache.org/jira/browse/NIFI-7621
> Project: Apache NiFi
> Issue Type: Task
> Components: Documentation Website, Tools and Build
>Affects Versions: 1.11.4
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Major
> Labels: terminology
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Change references to {{master}} branch to reflect new name {{main}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-7635) Fix StandardConfigurationContext to read the default property value from its ComponentNode instance
[ https://issues.apache.org/jira/browse/NIFI-7635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157521#comment-17157521 ] ASF subversion and git services commented on NIFI-7635: --- Commit 4f11e3626093d3090f97c0efc5e229d83b6006e4 in nifi's branch refs/heads/main from Peter Gyori [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=4f11e36 ] NIFI-7635: StandardConfigurationContext.getProperty() gets the property descriptor and its default value from the component itself This closes #4408. Signed-off-by: Mark Payne > Fix StandardConfigurationContext to read the default property value from its > ComponentNode instance > --- > > Key: NIFI-7635 > URL: https://issues.apache.org/jira/browse/NIFI-7635 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Peter Gyori >Assignee: Peter Gyori >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > If an effective property value is not set, > StandardConfigurationContext.getProperty(PropertyDescriptor property) reads > the default property value from its parameter (property) instead of the > ComponentNode instance. This results in faulty behavior when e.g. a > controller service is enabled with its default settings and one or more of > these default values differ from the default values of the supplied > PropertyDescriptor. > A fix needs to be implemented to get the property descriptor and its default > value from the component itself and not the supplied PropertyDescriptor > parameter. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] markap14 commented on pull request #4408: NIFI-7635: StandardConfigurationContext.getProperty() gets the proper…
markap14 commented on pull request #4408: URL: https://github.com/apache/nifi/pull/4408#issuecomment-658307295 Thanks @pgyori ! Code looks good. Verified that all tests pass, including System Tests. +1 merged to master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 closed pull request #4408: NIFI-7635: StandardConfigurationContext.getProperty() gets the proper…
markap14 closed pull request #4408: URL: https://github.com/apache/nifi/pull/4408 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-7633) Provide Process Groups with a FlowFile Concurrency level for transferring a batch of FlowFiles at once
[ https://issues.apache.org/jira/browse/NIFI-7633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-7633: - Issue Type: New Feature (was: Bug) > Provide Process Groups with a FlowFile Concurrency level for transferring a > batch of FlowFiles at once > -- > > Key: NIFI-7633 > URL: https://issues.apache.org/jira/browse/NIFI-7633 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.12.0 > > Time Spent: 10m > Remaining Estimate: 0h > > NIFI-7476 provides the ability for a Process Group to bring in a single > FlowFile at a time instead of continually streaming data. It also provides > the ability to output data as a batch. If a Process Group outputs the data in > a batch, another Process Group should have the ability to pull in all of that > data in a single batch as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7600) Defining JMS attribute types breaks due to invalid characters
[ https://issues.apache.org/jira/browse/NIFI-7600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-7600: - Status: Patch Available (was: Open) > Defining JMS attribute types breaks due to invalid characters > - > > Key: NIFI-7600 > URL: https://issues.apache.org/jira/browse/NIFI-7600 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.11.4 >Reporter: Darren Hitchman >Assignee: Darren Hitchman >Priority: Major > Labels: JMS > Fix For: 1.12.0 > > Original Estimate: 4h > Time Spent: 40m > Remaining Estimate: 3h 20m > > JMSPublisher causes Exception when setting attributes in the JMS message due > to invalid characters. When defining types to be used when sending JMS > attributes (e.g. JMS_IBM_Encoding.type=integer) , the JMSPublishercorrectly > sets the type but also adds the '.type' attribute into the message, causing > an Exception as JMS doesnt support a dot character in attribute names. > NIFI-6957 was suppose to fix this however the change in JMSPublish was > removed. > There is an option to 'allowIllegalChars' in PublishJMS however '.type' > attributes are passed along regardless of this setting. > > The JMSPublisher should ignore message attributes ending with '.type' when > there is a matching attribute without the '.type' suffix > > example exception. > > com.ibm.msg.client.jms.DetailedMessageFormatException: JMSCC0049: The > property name 'JMS_IBM_Encoding.type' is not a valid Java(tm) identifier. > The supplied property name does not conform to the allowed format described > in the JMS specification. > Check the characters used in the property name and modify as necessary. > at > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at > java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490) > at > com.ibm.msg.client.commonservices.j2se.NLSServices.createException(NLSServices.java:319) > at > com.ibm.msg.client.commonservices.nls.NLSServices.createException(NLSServices.java:226) > at > com.ibm.msg.client.jms.internal.JmsErrorUtils.createException(JmsErrorUtils.java:126) > at > com.ibm.msg.client.jms.internal.JmsMessageImpl.checkPropName(JmsMessageImpl.java:2595) > at > com.ibm.msg.client.jms.internal.JmsMessageImpl.setStringProperty(JmsMessageImpl.java:1997) > at > com.ibm.jms.JMSMessage.setStringProperty(JMSMessage.java:1516) > at > jdk.internal.reflect.GeneratedMethodAccessor198.invoke(Unknown Source) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:566) > at > org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:254) > at > org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler$ProxiedReturnObjectInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:240) > at com.sun.proxy.$Proxy145.setStringProperty(Unknown Source) > at > org.apache.nifi.jms.processors.JMSPublisher$JmsPropertySetterEnum.lambda$static$7(JMSPublisher.java:196) > at > org.apache.nifi.jms.processors.JMSPublisher$JmsPropertySetterEnum.setProperty(JMSPublisher.java:205) > at > org.apache.nifi.jms.processors.JMSPublisher.setMessageHeaderAndProperties(JMSPublisher.java:123) > at > org.apache.nifi.jms.processors.JMSPublisher$1.createMessage(JMSPublisher.java:59) > at > org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:593) > at > org.springframework.jms.core.JmsTemplate$4.doInJms(JmsTemplate.java:574) > at > org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:484) > at > org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:570) > at > org.apache.nifi.jms.processors.JMSPublisher.publish(JMSPublisher.java:54) > at > org.apache.nifi.jms.processors.PublishJMS.rendezvousWithJms(PublishJMS.java:214) > at > org.apache.nifi.jms.processors.PublishJMS.rendezvousWithJms(PublishJMS.java:68)
[jira] [Assigned] (NIFI-7493) XML Schema Inference can infer a type of String when it should be Record
[
https://issues.apache.org/jira/browse/NIFI-7493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne reassigned NIFI-7493:
Assignee: Mark Payne
> XML Schema Inference can infer a type of String when it should be Record
>
>
> Key: NIFI-7493
> URL: https://issues.apache.org/jira/browse/NIFI-7493
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.12.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> From the mailing list:
> {quote}I have configured a XMLReader to use the Infer Schema. The other issue
> is that I have problems converting sub records. My records looks something
> like this: John Doe
> some there
> workingman
> New York
> A Company
>
>
>
>
> The issues are with the subrecords in part 3. I have configured the XMLReader
> property "Field Name for Content" = value
>
> When the data is being converted via a XMLWriter the output for the
> additionalInfo fields looks like this:
> MapRecord[\{name=Location,
> value=New York}]
> MapRecord[\{name=Company, value=A
> Company}]
>
>
>
> If I use a JSONWriter I gets this:
> "Part3": { "Details": {
> "additionalInfo": [ "MapRecord[\{name=Location, value=New York}]",
> "MapRecord[\{name=Company, value=A Company}]" ]
> }
> }{quote}
> The issue appears to be that "additionalInfo" is being inferred as a String,
> but the XML Reader is returning a Record.
>
> This is probably because the "additionalInfo" element contains String
> content and no child nodes. However, it does have attributes. As a result,
> the XML Reader will return a Record. I'm guessing that attributes are not
> taken into account in the schema inference, though, and since
> "additionalInfo" has no child nodes but has textual content, it must be a
> String.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] joewitt commented on pull request #4396: NIFI-7621 & NIFI-7614 Updated terminology throughout codebase and doc…
joewitt commented on pull request #4396: URL: https://github.com/apache/nifi/pull/4396#issuecomment-658297061 +1 Please self merge. Not just why the osx build stopped but given the others went fine I suspect we're ok. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] joewitt commented on pull request #4354: NIFI-7304 Removed default value for nifi.web.max.content.size.
joewitt commented on pull request #4354: URL: https://github.com/apache/nifi/pull/4354#issuecomment-658294462 +1 please self-merge but reviewed and good to go. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #835: MINIFICPP-1281 - Improve test performance by using event polling instead of sleep by sync
fgerlits commented on a change in pull request #835:
URL: https://github.com/apache/nifi-minifi-cpp/pull/835#discussion_r454491636
##
File path: libminifi/include/utils/IntegrationTestUtils.h
##
@@ -0,0 +1,66 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+
+#include
+#include
+#include
+
+#include "../../../libminifi/test/TestBase.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace utils {
+
+template
+bool verifyEventHappenedInPollTime(const std::chrono::duration&
wait_duration, Fun&& check) {
+ std::chrono::system_clock::time_point wait_end =
std::chrono::system_clock::now() + wait_duration;
+ do {
+if (std::forward(check)()) {
+ return true;
+}
+std::this_thread::sleep_for(std::chrono::milliseconds(100));
+ } while (std::chrono::system_clock::now() < wait_end);
+ return false;
+}
+
+template
+bool verifyLogLinePresenceInPollTime(const std::chrono::duration&
wait_duration, String... patterns) {
+ // This helper is to be removed once we upgrade to support gcc 4.9+ only
Review comment:
Sorry for being dense, but if I find this comment later, after we have
upgraded to gcc >= 4.9 only, I'll have no idea how I am supposed to rewrite it.
Maybe include the better (post gcc-4.9) version of the code in the comment?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #835: MINIFICPP-1281 - Improve test performance by using event polling instead of sleep by sync
fgerlits commented on a change in pull request #835:
URL: https://github.com/apache/nifi-minifi-cpp/pull/835#discussion_r454491636
##
File path: libminifi/include/utils/IntegrationTestUtils.h
##
@@ -0,0 +1,66 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+
+#include
+#include
+#include
+
+#include "../../../libminifi/test/TestBase.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace utils {
+
+template
+bool verifyEventHappenedInPollTime(const std::chrono::duration&
wait_duration, Fun&& check) {
+ std::chrono::system_clock::time_point wait_end =
std::chrono::system_clock::now() + wait_duration;
+ do {
+if (std::forward(check)()) {
+ return true;
+}
+std::this_thread::sleep_for(std::chrono::milliseconds(100));
+ } while (std::chrono::system_clock::now() < wait_end);
+ return false;
+}
+
+template
+bool verifyLogLinePresenceInPollTime(const std::chrono::duration&
wait_duration, String... patterns) {
+ // This helper is to be removed once we upgrade to support gcc 4.9+ only
Review comment:
Sorry for being dense, but if I find this comment later, after we have
upgraded to gcc >= 4.9 only, I'll have no idea how I am supposed to rewrite the
code. Maybe include the better (post gcc-4.9) version of the code in the
comment?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Resolved] (NIFI-6128) UnpackContent has no awareness of file permissions
[ https://issues.apache.org/jira/browse/NIFI-6128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi resolved NIFI-6128. --- Fix Version/s: 1.12.0 Resolution: Fixed > UnpackContent has no awareness of file permissions > -- > > Key: NIFI-6128 > URL: https://issues.apache.org/jira/browse/NIFI-6128 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.9.0 >Reporter: Justin Rittenhouse >Assignee: Tamás Bunth >Priority: Major > Fix For: 1.12.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > MergeContent using TAR for packing allows for storing permissions in the > newly created tarball via the tar.permissions attribute on incoming files. > UnpackContent discards all permissions information on extraction. This makes > it impossible to restore a file from a tarball with previously recorded > permissions via NiFi. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6128) UnpackContent has no awareness of file permissions
[ https://issues.apache.org/jira/browse/NIFI-6128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157482#comment-17157482 ] ASF subversion and git services commented on NIFI-6128: --- Commit b1e77019accec8cb32c41d34838f2225299f72b4 in nifi's branch refs/heads/main from Tamás Bunth [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b1e7701 ] NIFI-6128 UnpackContent: Store unpacked file data Tar format allows us to archive files with their original permission, owner, group name and last modification time. When unpacking with Tar unpacker, these information are stored in new attributes with names: "file.inner.*". This way, it preserves backward compatibility when using parallel with GetFile processor (which stores information in "file.*"). NIFI-6128 Tar unpackContent: assert date of last modification of content is a valid date format. NIFI-6128 UnpackContent: use original attributes In case of tar format: - Use "file.*" attributes instead of "file.inner.*" (which eventually lead to overwrite if tar had been fetched with GetFile) - Store file permission in "rwx" format, instead of integer representation. - Also replace SimpleDateFormat with DateTimeFormatter. Replace generic error with IllegalArgumentException Also refactor: move permission string to top of file as static constant. Update test Remove 'file systems' text Unpackcontent: Fill file.creationTime attribute which always holds the very same value as file.lastModifiedTime. This closes #4370. Signed-off-by: Peter Turcsanyi > UnpackContent has no awareness of file permissions > -- > > Key: NIFI-6128 > URL: https://issues.apache.org/jira/browse/NIFI-6128 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.9.0 >Reporter: Justin Rittenhouse >Assignee: Tamás Bunth >Priority: Major > Time Spent: 2h 10m > Remaining Estimate: 0h > > MergeContent using TAR for packing allows for storing permissions in the > newly created tarball via the tar.permissions attribute on incoming files. > UnpackContent discards all permissions information on extraction. This makes > it impossible to restore a file from a tarball with previously recorded > permissions via NiFi. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6128) UnpackContent has no awareness of file permissions
[ https://issues.apache.org/jira/browse/NIFI-6128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157481#comment-17157481 ] ASF subversion and git services commented on NIFI-6128: --- Commit b1e77019accec8cb32c41d34838f2225299f72b4 in nifi's branch refs/heads/main from Tamás Bunth [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b1e7701 ] NIFI-6128 UnpackContent: Store unpacked file data Tar format allows us to archive files with their original permission, owner, group name and last modification time. When unpacking with Tar unpacker, these information are stored in new attributes with names: "file.inner.*". This way, it preserves backward compatibility when using parallel with GetFile processor (which stores information in "file.*"). NIFI-6128 Tar unpackContent: assert date of last modification of content is a valid date format. NIFI-6128 UnpackContent: use original attributes In case of tar format: - Use "file.*" attributes instead of "file.inner.*" (which eventually lead to overwrite if tar had been fetched with GetFile) - Store file permission in "rwx" format, instead of integer representation. - Also replace SimpleDateFormat with DateTimeFormatter. Replace generic error with IllegalArgumentException Also refactor: move permission string to top of file as static constant. Update test Remove 'file systems' text Unpackcontent: Fill file.creationTime attribute which always holds the very same value as file.lastModifiedTime. This closes #4370. Signed-off-by: Peter Turcsanyi > UnpackContent has no awareness of file permissions > -- > > Key: NIFI-6128 > URL: https://issues.apache.org/jira/browse/NIFI-6128 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.9.0 >Reporter: Justin Rittenhouse >Assignee: Tamás Bunth >Priority: Major > Time Spent: 2h 10m > Remaining Estimate: 0h > > MergeContent using TAR for packing allows for storing permissions in the > newly created tarball via the tar.permissions attribute on incoming files. > UnpackContent discards all permissions information on extraction. This makes > it impossible to restore a file from a tarball with previously recorded > permissions via NiFi. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #4370: NIFI-6128 UnpackContent: Store unpacked file data
asfgit closed pull request #4370: URL: https://github.com/apache/nifi/pull/4370 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-6128) UnpackContent has no awareness of file permissions
[ https://issues.apache.org/jira/browse/NIFI-6128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157480#comment-17157480 ] ASF subversion and git services commented on NIFI-6128: --- Commit b1e77019accec8cb32c41d34838f2225299f72b4 in nifi's branch refs/heads/main from Tamás Bunth [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b1e7701 ] NIFI-6128 UnpackContent: Store unpacked file data Tar format allows us to archive files with their original permission, owner, group name and last modification time. When unpacking with Tar unpacker, these information are stored in new attributes with names: "file.inner.*". This way, it preserves backward compatibility when using parallel with GetFile processor (which stores information in "file.*"). NIFI-6128 Tar unpackContent: assert date of last modification of content is a valid date format. NIFI-6128 UnpackContent: use original attributes In case of tar format: - Use "file.*" attributes instead of "file.inner.*" (which eventually lead to overwrite if tar had been fetched with GetFile) - Store file permission in "rwx" format, instead of integer representation. - Also replace SimpleDateFormat with DateTimeFormatter. Replace generic error with IllegalArgumentException Also refactor: move permission string to top of file as static constant. Update test Remove 'file systems' text Unpackcontent: Fill file.creationTime attribute which always holds the very same value as file.lastModifiedTime. This closes #4370. Signed-off-by: Peter Turcsanyi > UnpackContent has no awareness of file permissions > -- > > Key: NIFI-6128 > URL: https://issues.apache.org/jira/browse/NIFI-6128 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.9.0 >Reporter: Justin Rittenhouse >Assignee: Tamás Bunth >Priority: Major > Time Spent: 2h 10m > Remaining Estimate: 0h > > MergeContent using TAR for packing allows for storing permissions in the > newly created tarball via the tar.permissions attribute on incoming files. > UnpackContent discards all permissions information on extraction. This makes > it impossible to restore a file from a tarball with previously recorded > permissions via NiFi. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #835: MINIFICPP-1281 - Improve test performance by using event polling instead of sleep by sync
hunyadi-dev commented on a change in pull request #835:
URL: https://github.com/apache/nifi-minifi-cpp/pull/835#discussion_r454463047
##
File path: libminifi/test/integration/OnScheduleErrorHandlingTests.cpp
##
@@ -25,31 +25,45 @@
#include "core/state/ProcessorController.h"
#include "../TestBase.h"
#include "../KamikazeProcessor.h"
+#include "utils/IntegrationTestUtils.h"
/*Verify behavior in case exceptions are thrown in onSchedule or onTrigger
functions
* KamikazeProcessor is a test processor to trigger errors in these functions
*/
class KamikazeErrorHandlingTests : public IntegrationBase {
public:
void runAssertions() override {
-std::string logs = LogTestController::getInstance().log_output.str();
-
-auto result = countPatInStr(logs,
minifi::processors::KamikazeProcessor::OnScheduleExceptionStr);
-size_t last_pos = result.first;
-int occurrences = result.second;
-
-assert(occurrences > 1); // Verify retry of onSchedule and onUnSchedule
calls
+using org::apache::nifi::minifi::utils::verifyEventHappenedInPollTime;
+
assert(verifyEventHappenedInPollTime(std::chrono::milliseconds(wait_time_), [&]
{
+ const std::string logs =
LogTestController::getInstance().log_output.str();
+ const auto result = countPatInStr(logs,
minifi::processors::KamikazeProcessor::OnScheduleExceptionStr);
+ const int occurrences = result.second;
+ if (occurrences <= 1) { // Verify retry of onSchedule and onUnSchedule
calls
Review comment:
I like naming this occurences, it will probably get optimized anyway,
but I wouldn't know what it means otherwise. Also I almost always prefer
relation operators `<` and `<=` to `>` and `>=`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #835: MINIFICPP-1281 - Improve test performance by using event polling instead of sleep by sync
hunyadi-dev commented on a change in pull request #835:
URL: https://github.com/apache/nifi-minifi-cpp/pull/835#discussion_r454459865
##
File path: libminifi/include/utils/IntegrationTestUtils.h
##
@@ -0,0 +1,66 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+
+#include
+#include
+#include
+
+#include "../../../libminifi/test/TestBase.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace utils {
+
+template
+bool verifyEventHappenedInPollTime(const std::chrono::duration&
wait_duration, Fun&& check) {
+ std::chrono::system_clock::time_point wait_end =
std::chrono::system_clock::now() + wait_duration;
+ do {
+if (std::forward(check)()) {
+ return true;
+}
+std::this_thread::sleep_for(std::chrono::milliseconds(100));
+ } while (std::chrono::system_clock::now() < wait_end);
+ return false;
+}
+
+template
+bool verifyLogLinePresenceInPollTime(const std::chrono::duration&
wait_duration, String... patterns) {
+ // This helper is to be removed once we upgrade to support gcc 4.9+ only
+ std::vector pattern_list;
+ std::initializer_list {(pattern_list.push_back(patterns), 0)...};
+ auto check = [&] {
+const std::string logs = LogTestController::getInstance().log_output.str();
+for (const std::string& pattern : pattern_list) {
Review comment:
Yup, good idea.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #835: MINIFICPP-1281 - Improve test performance by using event polling instead of sleep by sync
hunyadi-dev commented on a change in pull request #835:
URL: https://github.com/apache/nifi-minifi-cpp/pull/835#discussion_r454460253
##
File path: libminifi/include/utils/IntegrationTestUtils.h
##
@@ -0,0 +1,66 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+
+#include
+#include
+#include
+
+#include "../../../libminifi/test/TestBase.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace utils {
+
+template
+bool verifyEventHappenedInPollTime(const std::chrono::duration&
wait_duration, Fun&& check) {
+ std::chrono::system_clock::time_point wait_end =
std::chrono::system_clock::now() + wait_duration;
+ do {
+if (std::forward(check)()) {
+ return true;
+}
+std::this_thread::sleep_for(std::chrono::milliseconds(100));
+ } while (std::chrono::system_clock::now() < wait_end);
+ return false;
+}
+
+template
+bool verifyLogLinePresenceInPollTime(const std::chrono::duration&
wait_duration, String... patterns) {
+ // This helper is to be removed once we upgrade to support gcc 4.9+ only
Review comment:
Ideally you should not need to unpact the arguments into a new vector.
However, unpacking does not properly work on gcc 4.8.5 when defining a variadic
check.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Resolved] (NIFI-7589) Unpack processor tar unpacker creates incorrect path value
[
https://issues.apache.org/jira/browse/NIFI-7589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi resolved NIFI-7589.
---
Fix Version/s: 1.12.0
Resolution: Fixed
> Unpack processor tar unpacker creates incorrect path value
> --
>
> Key: NIFI-7589
> URL: https://issues.apache.org/jira/browse/NIFI-7589
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Tamás Bunth
>Assignee: Tamás Bunth
>Priority: Minor
> Fix For: 1.12.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> *Steps to reproduce*:
> * Create a tarball with one file in it.
> * Create a flow:
> {code:java}
> Getfile -> UnpackContent -> logAttribute
> {code}
> ** Set Getfile "directory" to read the tarball
> ** Set UnpackContent "PackagingFormat" to tar.
> * Examine the 'path' flowfile attribute in the logs.
> *Expected results*:
> Since the file is in the tarball root, "path" attribute should be "." or "/".
> *Current results*:
> "null/"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-7589) Unpack processor tar unpacker creates incorrect path value
[
https://issues.apache.org/jira/browse/NIFI-7589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-7589:
--
Component/s: Extensions
> Unpack processor tar unpacker creates incorrect path value
> --
>
> Key: NIFI-7589
> URL: https://issues.apache.org/jira/browse/NIFI-7589
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Tamás Bunth
>Assignee: Tamás Bunth
>Priority: Minor
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> *Steps to reproduce*:
> * Create a tarball with one file in it.
> * Create a flow:
> {code:java}
> Getfile -> UnpackContent -> logAttribute
> {code}
> ** Set Getfile "directory" to read the tarball
> ** Set UnpackContent "PackagingFormat" to tar.
> * Examine the 'path' flowfile attribute in the logs.
> *Expected results*:
> Since the file is in the tarball root, "path" attribute should be "." or "/".
> *Current results*:
> "null/"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #4371: NIFI-7589 Fix path value when unpacking tar
asfgit closed pull request #4371: URL: https://github.com/apache/nifi/pull/4371 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-7589) Unpack processor tar unpacker creates incorrect path value
[
https://issues.apache.org/jira/browse/NIFI-7589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157463#comment-17157463
]
ASF subversion and git services commented on NIFI-7589:
---
Commit 72666a9c7e87c39110039ac38b4ea303c317330d in nifi's branch
refs/heads/main from Tamás Bunth
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=72666a9 ]
NIFI-7589 Fix path value when unpacking tar
Refactor: remove variable reassignment
This closes #4371.
Signed-off-by: Peter Turcsanyi
> Unpack processor tar unpacker creates incorrect path value
> --
>
> Key: NIFI-7589
> URL: https://issues.apache.org/jira/browse/NIFI-7589
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Tamás Bunth
>Assignee: Tamás Bunth
>Priority: Minor
> Time Spent: 1h
> Remaining Estimate: 0h
>
> *Steps to reproduce*:
> * Create a tarball with one file in it.
> * Create a flow:
> {code:java}
> Getfile -> UnpackContent -> logAttribute
> {code}
> ** Set Getfile "directory" to read the tarball
> ** Set UnpackContent "PackagingFormat" to tar.
> * Examine the 'path' flowfile attribute in the logs.
> *Expected results*:
> Since the file is in the tarball root, "path" attribute should be "." or "/".
> *Current results*:
> "null/"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] turcsanyip commented on pull request #4371: NIFI-7589 Fix path value when unpacking tar
turcsanyip commented on pull request #4371: URL: https://github.com/apache/nifi/pull/4371#issuecomment-658248961 Merging to main. @Wastack Thanks for fixing it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] arpadboda commented on a change in pull request #835: MINIFICPP-1281 - Improve test performance by using event polling instead of sleep by sync
arpadboda commented on a change in pull request #835:
URL: https://github.com/apache/nifi-minifi-cpp/pull/835#discussion_r454431840
##
File path: extensions/http-curl/tests/HttpGetIntegrationTest.cpp
##
@@ -74,6 +71,8 @@ class HttpResponder : public CivetHandler {
};
int main(int argc, char **argv) {
+ using org::apache::nifi::minifi::utils::verifyLogLinePresenceInPollTime;
Review comment:
I'm not sure this has anything to do in utils in case we only use it in
for verification. Should be part of a test namespace.
##
File path: extensions/http-curl/tests/C2FailedUpdateTest.cpp
##
@@ -201,13 +199,11 @@ int main(int argc, char **argv) {
controller->load();
controller->start();
- waitToVerifyProcessor();
+ const bool test_success =
verifyLogLinePresenceInPollTime(std::chrono::seconds(10), "Invalid
configuration payload", "update failed.");
Review comment:
Why the extra bool and why don't we do it in the proper overridden
function?
##
File path: libminifi/include/utils/IntegrationTestUtils.h
##
@@ -0,0 +1,66 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+
+#include
+#include
+#include
+
+#include "../../../libminifi/test/TestBase.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace utils {
+
+template
+bool verifyEventHappenedInPollTime(const std::chrono::duration&
wait_duration, Fun&& check) {
+ std::chrono::system_clock::time_point wait_end =
std::chrono::system_clock::now() + wait_duration;
+ do {
+if (std::forward(check)()) {
+ return true;
+}
+std::this_thread::sleep_for(std::chrono::milliseconds(100));
+ } while (std::chrono::system_clock::now() < wait_end);
+ return false;
+}
+
+template
+bool verifyLogLinePresenceInPollTime(const std::chrono::duration&
wait_duration, String... patterns) {
+ // This helper is to be removed once we upgrade to support gcc 4.9+ only
+ std::vector pattern_list;
+ std::initializer_list {(pattern_list.push_back(patterns), 0)...};
+ auto check = [&] {
+const std::string logs = LogTestController::getInstance().log_output.str();
+for (const std::string& pattern : pattern_list) {
Review comment:
I think std_all is quite talkative for such cases to make the purpose
clear:
```
std::all_of(pattern_list.cbegin(), pattern_list.cend(), [](const
std::string& pattern){ return logs.find(pattern) != std::string::npos; })
```
##
File path: libminifi/include/utils/IntegrationTestUtils.h
##
@@ -0,0 +1,66 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+
+#include
+#include
+#include
+
+#include "../../../libminifi/test/TestBase.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace utils {
+
+template
+bool verifyEventHappenedInPollTime(const std::chrono::duration&
wait_duration, Fun&& check) {
+ std::chrono::system_clock::time_point wait_end =
std::chrono::system_clock::now() + wait_duration;
+ do {
+if (std::forward(check)()) {
+ return true;
+}
+std::this_thread::sleep_for(std::chrono::milliseconds(100));
+ } while (std::chrono::system_clock::now() < wait_end);
+ return false;
+}
+
+template
+bool verifyLogLinePresenceInPollTime(const std::chrono::duration&
wait_duration, String... patterns) {
+ // This helper is to be removed once we upgrade to support gcc
[GitHub] [nifi-minifi-cpp] arpadboda commented on pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor
arpadboda commented on pull request #821: URL: https://github.com/apache/nifi-minifi-cpp/pull/821#issuecomment-658232449 I still don't see any properties being required, otherwise looks good. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-registry] kevdoran commented on pull request #287: NIFIREG-399 Support chown docker in docker
kevdoran commented on pull request #287:
URL: https://github.com/apache/nifi-registry/pull/287#issuecomment-658231312
@nkininge this is failing for me when I do a normal build (outside of
docker):
```
[INFO] dockermaven FAILURE [ 51.351
s]
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 05:06 min
[INFO] Finished at: 2020-07-14T10:55:04-04:00
[INFO]
[ERROR] Failed to execute goal
com.spotify:dockerfile-maven-plugin:1.3.5:build (default) on project
dockermaven: Could not build image: The command '/bin/sh -c groupadd -g ${GID}
nifi || groupmod -n nifi `getent group ${GID} | cut -d: -f1` && useradd
--shell /bin/bash -u ${UID} -g ${GID} -m nifi && chown -R nifi:nifi
${NIFI_REGISTRY_BASE_DIR} && apt-get update && apt-get install -y jq
xmlstarlet procps' returned a non-zero code: 1 -> [Help 1]
```
Looks like we'll have to explore a different solution
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (MINIFICPP-1293) PropertyTests fails on Windows in time zones east of Greenwich
[
https://issues.apache.org/jira/browse/MINIFICPP-1293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ferenc Gerlits updated MINIFICPP-1293:
--
Description:
In the "Test DateTime Conversion" in PropertyTests.cpp, the test
{noformat}
int64_t timestamp = 0LL;
REQUIRE(true ==
org::apache::nifi::minifi::core::Property::StringToDateTime("1970-01-01T00:00:00Z",
timestamp));
REQUIRE(0LL == timestamp);
{noformat}
fails, because {{StringToDateTime}} uses {{mktime}} internally to convert a UTC
{{stuct tm}} to a localtime {{time_h}}, and then adjust it back to UTC by
adding a time zone offset.
"1970-01-01T00:00:00" in local time east of Greenwich results in a negative
{{time_h}} value, which is not officially supported by {{mktime}}. On Windows,
{{mktime}} returns -1 to signal an error, so the test fails. On Linux, if the
time zone is UTC+1, {{mktime}} returns -3600, so the test passes; however, if
we were to test "1970-01-01T00:59:59", then {{mktime}} would have to return -1
as the answer, which looks like an error, so this test would fail.
was:
In the "Test DateTime Conversion" in PropertyTests.cpp, the test
{noformat}
int64_t timestamp = 0LL;
REQUIRE(true ==
org::apache::nifi::minifi::core::Property::StringToDateTime("1970-01-01T00:00:00Z",
timestamp));
REQUIRE(0LL == timestamp);
{noformat}
fails, because {{StringToDateTime}} uses {{mktime}} internally to convert a UTC
{{stuct tm}} to a localtime {{time_h}}, and then adjust it back to UTC by
adding a time zone offset.
"1970-01-01T00:00:00Z" in local time east of Greenwich results in a negative
{{time_h}} value, which is not officially supported by {{mktime}}. On Windows,
{{mktime}} returns -1 to signal an error, so the test fails. On Linux, if the
time zone is UTC+1, {{mktime}} returns -3600, so the test passes; however, if
we were to test "1970-01-01T00:59:59Z", then {{mktime}} would have to return -1
as the answer, which looks like an error, so this test would fail.
> PropertyTests fails on Windows in time zones east of Greenwich
> --
>
> Key: MINIFICPP-1293
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1293
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Ferenc Gerlits
>Assignee: Ferenc Gerlits
>Priority: Minor
> Fix For: 0.8.0
>
>
> In the "Test DateTime Conversion" in PropertyTests.cpp, the test
> {noformat}
> int64_t timestamp = 0LL;
> REQUIRE(true ==
> org::apache::nifi::minifi::core::Property::StringToDateTime("1970-01-01T00:00:00Z",
> timestamp));
> REQUIRE(0LL == timestamp);
> {noformat}
> fails, because {{StringToDateTime}} uses {{mktime}} internally to convert a
> UTC {{stuct tm}} to a localtime {{time_h}}, and then adjust it back to UTC by
> adding a time zone offset.
> "1970-01-01T00:00:00" in local time east of Greenwich results in a negative
> {{time_h}} value, which is not officially supported by {{mktime}}. On
> Windows, {{mktime}} returns -1 to signal an error, so the test fails. On
> Linux, if the time zone is UTC+1, {{mktime}} returns -3600, so the test
> passes; however, if we were to test "1970-01-01T00:59:59", then {{mktime}}
> would have to return -1 as the answer, which looks like an error, so this
> test would fail.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor
hunyadi-dev commented on a change in pull request #821:
URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r454397770
##
File path: libminifi/include/core/PropertyValidation.h
##
@@ -259,6 +259,22 @@ class UnsignedLongValidator : public PropertyValidator {
}
};
+class NonBlankValidator : public PropertyValidator {
+ public:
+ explicit NonBlankValidator(const std::string& name)
+ : PropertyValidator(name) {
+ }
+ ~NonBlankValidator() override = default;
+
+ ValidationResult validate(const std::string& subject, const
std::shared_ptr& input) const final {
+return validate(subject, input->getStringValue());
Review comment:
I thought yaml parsing ignored padding, so
```yaml
name: A
```
was the same as
```yaml
name: A
```
Will add trimming for the string.
##
File path: libminifi/include/core/PropertyValidation.h
##
@@ -259,6 +259,22 @@ class UnsignedLongValidator : public PropertyValidator {
}
};
+class NonBlankValidator : public PropertyValidator {
+ public:
+ explicit NonBlankValidator(const std::string& name)
+ : PropertyValidator(name) {
+ }
+ ~NonBlankValidator() override = default;
+
+ ValidationResult validate(const std::string& subject, const
std::shared_ptr& input) const final {
+return validate(subject, input->getStringValue());
Review comment:
I thought yaml parsing ignored padding, so
```yaml
name: A
```
was the same as
```yaml
name: A
```
**Will add trimming for the string.**
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor
hunyadi-dev commented on a change in pull request #821:
URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r454397770
##
File path: libminifi/include/core/PropertyValidation.h
##
@@ -259,6 +259,22 @@ class UnsignedLongValidator : public PropertyValidator {
}
};
+class NonBlankValidator : public PropertyValidator {
+ public:
+ explicit NonBlankValidator(const std::string& name)
+ : PropertyValidator(name) {
+ }
+ ~NonBlankValidator() override = default;
+
+ ValidationResult validate(const std::string& subject, const
std::shared_ptr& input) const final {
+return validate(subject, input->getStringValue());
Review comment:
I thought yaml parsing ignored padding, so
```yaml
name: A
```
was the same as
```yaml
name: A
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] arpadboda commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor
arpadboda commented on a change in pull request #821:
URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r454393012
##
File path: libminifi/include/core/PropertyValidation.h
##
@@ -259,6 +259,22 @@ class UnsignedLongValidator : public PropertyValidator {
}
};
+class NonBlankValidator : public PropertyValidator {
+ public:
+ explicit NonBlankValidator(const std::string& name)
+ : PropertyValidator(name) {
+ }
+ ~NonBlankValidator() override = default;
+
+ ValidationResult validate(const std::string& subject, const
std::shared_ptr& input) const final {
+return validate(subject, input->getStringValue());
Review comment:
How would it validate not being blank?
I think we should trim both left and right and check if the result is empty.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor
hunyadi-dev commented on a change in pull request #821:
URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r454388997
##
File path: extensions/standard-processors/processors/RetryFlowFile.cpp
##
@@ -0,0 +1,183 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "RetryFlowFile.h"
+
+#include "core/PropertyValidation.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property
RetryFlowFile::RetryAttribute(core::PropertyBuilder::createProperty("Retry
Attribute")
+->withDescription(
+"The name of the attribute that contains the current retry count for
the FlowFile."
+"WARNING: If the name matches an attribute already on the FlowFile
that does not contain a numerical value, "
+"the processor will either overwrite that attribute with '1' or fail
based on configuration.")
+->withDefaultValue("flowfile.retries")
Review comment:
Added non-blank validator.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor
hunyadi-dev commented on a change in pull request #821:
URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r454373723
##
File path: extensions/standard-processors/processors/RetryFlowFile.cpp
##
@@ -0,0 +1,183 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "RetryFlowFile.h"
+
+#include "core/PropertyValidation.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property
RetryFlowFile::RetryAttribute(core::PropertyBuilder::createProperty("Retry
Attribute")
+->withDescription(
+"The name of the attribute that contains the current retry count for
the FlowFile."
+"WARNING: If the name matches an attribute already on the FlowFile
that does not contain a numerical value, "
+"the processor will either overwrite that attribute with '1' or fail
based on configuration.")
+->withDefaultValue("flowfile.retries")
+->supportsExpressionLanguage(true)
+->build());
+
+core::Property
RetryFlowFile::MaximumRetries(core::PropertyBuilder::createProperty("Maximum
Retries")
+->withDescription("The maximum number of times a FlowFile can be retried
before being passed to the 'retries_exceeded' relationship.")
+->withDefaultValue(3)
+->supportsExpressionLanguage(true)
+->build());
+
+core::Property
RetryFlowFile::PenalizeRetries(core::PropertyBuilder::createProperty("Penalize
Retries")
+ ->withDescription("If set to 'true', this Processor will penalize input
FlowFiles before passing them to the 'retry' relationship. This does not apply
to the 'retries_exceeded' relationship.")
+ ->withDefaultValue(true)
+ ->build());
+
+core::Property
RetryFlowFile::FailOnNonNumericalOverwrite(core::PropertyBuilder::createProperty("Fail
on Non-numerical Overwrite")
+->withDescription("If the FlowFile already has the attribute defined in
'Retry Attribute' that is *not* a number, fail the FlowFile instead of
resetting that value to '1'")
+->withDefaultValue(false)
+->build());
+
+core::Property
RetryFlowFile::ReuseMode(core::PropertyBuilder::createProperty("Reuse Mode")
+->withDescription(
+"Defines how the Processor behaves if the retry FlowFile has a
different retry UUID than "
+"the instance that received the FlowFile. This generally means that
the attribute was "
+"not reset after being successfully retried by a previous instance of
this processor.")
+->withAllowableValues({FAIL_ON_REUSE, WARN_ON_REUSE,
RESET_REUSE})
+->withDefaultValue(FAIL_ON_REUSE)
+->build());
+
+core::Relationship RetryFlowFile::Retry("retry",
+ "Input FlowFile has not exceeded the configured maximum retry count, pass
this relationship back to the input Processor to create a limited feedback
loop.");
+core::Relationship RetryFlowFile::RetriesExceeded("retries_exceeded",
+ "Input FlowFile has exceeded the configured maximum retry count, do not pass
this relationship back to the input Processor to terminate the limited feedback
loop.");
+core::Relationship RetryFlowFile::Failure("failure",
+"The processor is configured such that a non-numerical value on 'Retry
Attribute' results in a failure instead of resetting "
+"that value to '1'. This will immediately terminate the limited feedback
loop. Might also include when 'Maximum Retries' contains "
+" attribute expression language that does not resolve to an Integer.");
+
+void RetryFlowFile::initialize() {
+ setSupportedProperties({
+RetryAttribute,
+MaximumRetries,
+PenalizeRetries,
+FailOnNonNumericalOverwrite,
+ReuseMode,
+ });
+ setSupportedRelationships({
+Retry,
+RetriesExceeded,
+Failure,
+ });
+}
+
+void RetryFlowFile::onSchedule(core::ProcessContext* context,
core::ProcessSessionFactory* /* sessionFactory */) {
+ context->getProperty(RetryAttribute.getName(), retry_attribute_);
+ context->getProperty(MaximumRetries.getName(), maximum_retries_);
+ context->getProperty(PenalizeRetries.getName(), penalize_retries_);
+ context->getProperty(FailOnNonNumericalOverwrite.getName(),
fail_on_non_numerical_overwrite_);
+ context->getProperty(ReuseMode.getName(),
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor
hunyadi-dev commented on a change in pull request #821:
URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r454373723
##
File path: extensions/standard-processors/processors/RetryFlowFile.cpp
##
@@ -0,0 +1,183 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "RetryFlowFile.h"
+
+#include "core/PropertyValidation.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property
RetryFlowFile::RetryAttribute(core::PropertyBuilder::createProperty("Retry
Attribute")
+->withDescription(
+"The name of the attribute that contains the current retry count for
the FlowFile."
+"WARNING: If the name matches an attribute already on the FlowFile
that does not contain a numerical value, "
+"the processor will either overwrite that attribute with '1' or fail
based on configuration.")
+->withDefaultValue("flowfile.retries")
+->supportsExpressionLanguage(true)
+->build());
+
+core::Property
RetryFlowFile::MaximumRetries(core::PropertyBuilder::createProperty("Maximum
Retries")
+->withDescription("The maximum number of times a FlowFile can be retried
before being passed to the 'retries_exceeded' relationship.")
+->withDefaultValue(3)
+->supportsExpressionLanguage(true)
+->build());
+
+core::Property
RetryFlowFile::PenalizeRetries(core::PropertyBuilder::createProperty("Penalize
Retries")
+ ->withDescription("If set to 'true', this Processor will penalize input
FlowFiles before passing them to the 'retry' relationship. This does not apply
to the 'retries_exceeded' relationship.")
+ ->withDefaultValue(true)
+ ->build());
+
+core::Property
RetryFlowFile::FailOnNonNumericalOverwrite(core::PropertyBuilder::createProperty("Fail
on Non-numerical Overwrite")
+->withDescription("If the FlowFile already has the attribute defined in
'Retry Attribute' that is *not* a number, fail the FlowFile instead of
resetting that value to '1'")
+->withDefaultValue(false)
+->build());
+
+core::Property
RetryFlowFile::ReuseMode(core::PropertyBuilder::createProperty("Reuse Mode")
+->withDescription(
+"Defines how the Processor behaves if the retry FlowFile has a
different retry UUID than "
+"the instance that received the FlowFile. This generally means that
the attribute was "
+"not reset after being successfully retried by a previous instance of
this processor.")
+->withAllowableValues({FAIL_ON_REUSE, WARN_ON_REUSE,
RESET_REUSE})
+->withDefaultValue(FAIL_ON_REUSE)
+->build());
+
+core::Relationship RetryFlowFile::Retry("retry",
+ "Input FlowFile has not exceeded the configured maximum retry count, pass
this relationship back to the input Processor to create a limited feedback
loop.");
+core::Relationship RetryFlowFile::RetriesExceeded("retries_exceeded",
+ "Input FlowFile has exceeded the configured maximum retry count, do not pass
this relationship back to the input Processor to terminate the limited feedback
loop.");
+core::Relationship RetryFlowFile::Failure("failure",
+"The processor is configured such that a non-numerical value on 'Retry
Attribute' results in a failure instead of resetting "
+"that value to '1'. This will immediately terminate the limited feedback
loop. Might also include when 'Maximum Retries' contains "
+" attribute expression language that does not resolve to an Integer.");
+
+void RetryFlowFile::initialize() {
+ setSupportedProperties({
+RetryAttribute,
+MaximumRetries,
+PenalizeRetries,
+FailOnNonNumericalOverwrite,
+ReuseMode,
+ });
+ setSupportedRelationships({
+Retry,
+RetriesExceeded,
+Failure,
+ });
+}
+
+void RetryFlowFile::onSchedule(core::ProcessContext* context,
core::ProcessSessionFactory* /* sessionFactory */) {
+ context->getProperty(RetryAttribute.getName(), retry_attribute_);
+ context->getProperty(MaximumRetries.getName(), maximum_retries_);
+ context->getProperty(PenalizeRetries.getName(), penalize_retries_);
+ context->getProperty(FailOnNonNumericalOverwrite.getName(),
fail_on_non_numerical_overwrite_);
+ context->getProperty(ReuseMode.getName(),
[jira] [Created] (MINIFICPP-1293) PropertyTests fails on Windows in time zones east of Greenwich
Ferenc Gerlits created MINIFICPP-1293:
-
Summary: PropertyTests fails on Windows in time zones east of
Greenwich
Key: MINIFICPP-1293
URL: https://issues.apache.org/jira/browse/MINIFICPP-1293
Project: Apache NiFi MiNiFi C++
Issue Type: Bug
Reporter: Ferenc Gerlits
Assignee: Ferenc Gerlits
Fix For: 0.8.0
In the "Test DateTime Conversion" in PropertyTests.cpp, the test
{noformat}
int64_t timestamp = 0LL;
REQUIRE(true ==
org::apache::nifi::minifi::core::Property::StringToDateTime("1970-01-01T00:00:00Z",
timestamp));
REQUIRE(0LL == timestamp);
{noformat}
fails, because {{StringToDateTime}} uses {{mktime}} internally to convert a UTC
{{stuct tm}} to a localtime {{time_h}}, and then adjust it back to UTC by
adding a time zone offset.
"1970-01-01T00:00:00Z" in local time east of Greenwich results in a negative
{{time_h}} value, which is not officially supported by {{mktime}}. On Windows,
{{mktime}} returns -1 to signal an error, so the test fails. On Linux, if the
time zone is UTC+1, {{mktime}} returns -3600, so the test passes; however, if
we were to test "1970-01-01T00:59:59Z", then {{mktime}} would have to return -1
as the answer, which looks like an error, so this test would fail.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-7635) Fix StandardConfigurationContext to read the default property value from its ComponentNode instance
[ https://issues.apache.org/jira/browse/NIFI-7635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Gyori updated NIFI-7635: -- Status: Patch Available (was: In Progress) > Fix StandardConfigurationContext to read the default property value from its > ComponentNode instance > --- > > Key: NIFI-7635 > URL: https://issues.apache.org/jira/browse/NIFI-7635 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Peter Gyori >Assignee: Peter Gyori >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > If an effective property value is not set, > StandardConfigurationContext.getProperty(PropertyDescriptor property) reads > the default property value from its parameter (property) instead of the > ComponentNode instance. This results in faulty behavior when e.g. a > controller service is enabled with its default settings and one or more of > these default values differ from the default values of the supplied > PropertyDescriptor. > A fix needs to be implemented to get the property descriptor and its default > value from the component itself and not the supplied PropertyDescriptor > parameter. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] pgyori opened a new pull request #4408: NIFI-7635: StandardConfigurationContext.getProperty() gets the proper…
pgyori opened a new pull request #4408:
URL: https://github.com/apache/nifi/pull/4408
…ty descriptor and its default value from the component itself
https://issues.apache.org/jira/browse/NIFI-7635
Description of PR
Fixes bug NIFI-7635.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `master`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni opened a new pull request #839: WIP: Refactor
adamdebreceni opened a new pull request #839: URL: https://github.com/apache/nifi-minifi-cpp/pull/839 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor
adamdebreceni commented on a change in pull request #821:
URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r454328316
##
File path: extensions/standard-processors/processors/RetryFlowFile.cpp
##
@@ -0,0 +1,183 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "RetryFlowFile.h"
+
+#include "core/PropertyValidation.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property
RetryFlowFile::RetryAttribute(core::PropertyBuilder::createProperty("Retry
Attribute")
+->withDescription(
+"The name of the attribute that contains the current retry count for
the FlowFile."
+"WARNING: If the name matches an attribute already on the FlowFile
that does not contain a numerical value, "
+"the processor will either overwrite that attribute with '1' or fail
based on configuration.")
+->withDefaultValue("flowfile.retries")
+->supportsExpressionLanguage(true)
+->build());
+
+core::Property
RetryFlowFile::MaximumRetries(core::PropertyBuilder::createProperty("Maximum
Retries")
+->withDescription("The maximum number of times a FlowFile can be retried
before being passed to the 'retries_exceeded' relationship.")
+->withDefaultValue(3)
+->supportsExpressionLanguage(true)
+->build());
+
+core::Property
RetryFlowFile::PenalizeRetries(core::PropertyBuilder::createProperty("Penalize
Retries")
+ ->withDescription("If set to 'true', this Processor will penalize input
FlowFiles before passing them to the 'retry' relationship. This does not apply
to the 'retries_exceeded' relationship.")
+ ->withDefaultValue(true)
+ ->build());
+
+core::Property
RetryFlowFile::FailOnNonNumericalOverwrite(core::PropertyBuilder::createProperty("Fail
on Non-numerical Overwrite")
+->withDescription("If the FlowFile already has the attribute defined in
'Retry Attribute' that is *not* a number, fail the FlowFile instead of
resetting that value to '1'")
+->withDefaultValue(false)
+->build());
+
+core::Property
RetryFlowFile::ReuseMode(core::PropertyBuilder::createProperty("Reuse Mode")
+->withDescription(
+"Defines how the Processor behaves if the retry FlowFile has a
different retry UUID than "
+"the instance that received the FlowFile. This generally means that
the attribute was "
+"not reset after being successfully retried by a previous instance of
this processor.")
+->withAllowableValues({FAIL_ON_REUSE, WARN_ON_REUSE,
RESET_REUSE})
+->withDefaultValue(FAIL_ON_REUSE)
+->build());
+
+core::Relationship RetryFlowFile::Retry("retry",
+ "Input FlowFile has not exceeded the configured maximum retry count, pass
this relationship back to the input Processor to create a limited feedback
loop.");
+core::Relationship RetryFlowFile::RetriesExceeded("retries_exceeded",
+ "Input FlowFile has exceeded the configured maximum retry count, do not pass
this relationship back to the input Processor to terminate the limited feedback
loop.");
+core::Relationship RetryFlowFile::Failure("failure",
+"The processor is configured such that a non-numerical value on 'Retry
Attribute' results in a failure instead of resetting "
+"that value to '1'. This will immediately terminate the limited feedback
loop. Might also include when 'Maximum Retries' contains "
+" attribute expression language that does not resolve to an Integer.");
+
+void RetryFlowFile::initialize() {
+ setSupportedProperties({
+RetryAttribute,
+MaximumRetries,
+PenalizeRetries,
+FailOnNonNumericalOverwrite,
+ReuseMode,
+ });
+ setSupportedRelationships({
+Retry,
+RetriesExceeded,
+Failure,
+ });
+}
+
+void RetryFlowFile::onSchedule(core::ProcessContext* context,
core::ProcessSessionFactory* /* sessionFactory */) {
+ context->getProperty(RetryAttribute.getName(), retry_attribute_);
+ context->getProperty(MaximumRetries.getName(), maximum_retries_);
+ context->getProperty(PenalizeRetries.getName(), penalize_retries_);
+ context->getProperty(FailOnNonNumericalOverwrite.getName(),
fail_on_non_numerical_overwrite_);
+ context->getProperty(ReuseMode.getName(),
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor
adamdebreceni commented on a change in pull request #821:
URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r454328316
##
File path: extensions/standard-processors/processors/RetryFlowFile.cpp
##
@@ -0,0 +1,183 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "RetryFlowFile.h"
+
+#include "core/PropertyValidation.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property
RetryFlowFile::RetryAttribute(core::PropertyBuilder::createProperty("Retry
Attribute")
+->withDescription(
+"The name of the attribute that contains the current retry count for
the FlowFile."
+"WARNING: If the name matches an attribute already on the FlowFile
that does not contain a numerical value, "
+"the processor will either overwrite that attribute with '1' or fail
based on configuration.")
+->withDefaultValue("flowfile.retries")
+->supportsExpressionLanguage(true)
+->build());
+
+core::Property
RetryFlowFile::MaximumRetries(core::PropertyBuilder::createProperty("Maximum
Retries")
+->withDescription("The maximum number of times a FlowFile can be retried
before being passed to the 'retries_exceeded' relationship.")
+->withDefaultValue(3)
+->supportsExpressionLanguage(true)
+->build());
+
+core::Property
RetryFlowFile::PenalizeRetries(core::PropertyBuilder::createProperty("Penalize
Retries")
+ ->withDescription("If set to 'true', this Processor will penalize input
FlowFiles before passing them to the 'retry' relationship. This does not apply
to the 'retries_exceeded' relationship.")
+ ->withDefaultValue(true)
+ ->build());
+
+core::Property
RetryFlowFile::FailOnNonNumericalOverwrite(core::PropertyBuilder::createProperty("Fail
on Non-numerical Overwrite")
+->withDescription("If the FlowFile already has the attribute defined in
'Retry Attribute' that is *not* a number, fail the FlowFile instead of
resetting that value to '1'")
+->withDefaultValue(false)
+->build());
+
+core::Property
RetryFlowFile::ReuseMode(core::PropertyBuilder::createProperty("Reuse Mode")
+->withDescription(
+"Defines how the Processor behaves if the retry FlowFile has a
different retry UUID than "
+"the instance that received the FlowFile. This generally means that
the attribute was "
+"not reset after being successfully retried by a previous instance of
this processor.")
+->withAllowableValues({FAIL_ON_REUSE, WARN_ON_REUSE,
RESET_REUSE})
+->withDefaultValue(FAIL_ON_REUSE)
+->build());
+
+core::Relationship RetryFlowFile::Retry("retry",
+ "Input FlowFile has not exceeded the configured maximum retry count, pass
this relationship back to the input Processor to create a limited feedback
loop.");
+core::Relationship RetryFlowFile::RetriesExceeded("retries_exceeded",
+ "Input FlowFile has exceeded the configured maximum retry count, do not pass
this relationship back to the input Processor to terminate the limited feedback
loop.");
+core::Relationship RetryFlowFile::Failure("failure",
+"The processor is configured such that a non-numerical value on 'Retry
Attribute' results in a failure instead of resetting "
+"that value to '1'. This will immediately terminate the limited feedback
loop. Might also include when 'Maximum Retries' contains "
+" attribute expression language that does not resolve to an Integer.");
+
+void RetryFlowFile::initialize() {
+ setSupportedProperties({
+RetryAttribute,
+MaximumRetries,
+PenalizeRetries,
+FailOnNonNumericalOverwrite,
+ReuseMode,
+ });
+ setSupportedRelationships({
+Retry,
+RetriesExceeded,
+Failure,
+ });
+}
+
+void RetryFlowFile::onSchedule(core::ProcessContext* context,
core::ProcessSessionFactory* /* sessionFactory */) {
+ context->getProperty(RetryAttribute.getName(), retry_attribute_);
+ context->getProperty(MaximumRetries.getName(), maximum_retries_);
+ context->getProperty(PenalizeRetries.getName(), penalize_retries_);
+ context->getProperty(FailOnNonNumericalOverwrite.getName(),
fail_on_non_numerical_overwrite_);
+ context->getProperty(ReuseMode.getName(),
[jira] [Created] (NIFI-7635) Fix StandardConfigurationContext to read the default property value from its ComponentNode instance
Peter Gyori created NIFI-7635: - Summary: Fix StandardConfigurationContext to read the default property value from its ComponentNode instance Key: NIFI-7635 URL: https://issues.apache.org/jira/browse/NIFI-7635 Project: Apache NiFi Issue Type: Bug Components: Core Framework Reporter: Peter Gyori Assignee: Peter Gyori If an effective property value is not set, StandardConfigurationContext.getProperty(PropertyDescriptor property) reads the default property value from its parameter (property) instead of the ComponentNode instance. This results in faulty behavior when e.g. a controller service is enabled with its default settings and one or more of these default values differ from the default values of the supplied PropertyDescriptor. A fix needs to be implemented to get the property descriptor and its default value from the component itself and not the supplied PropertyDescriptor parameter. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] fgerlits opened a new pull request #838: MINIFICPP-1183 Fix Windows compile errors
fgerlits opened a new pull request #838: URL: https://github.com/apache/nifi-minifi-cpp/pull/838 * Add () around std::max to work around Windows.h defining max as a macro; * Move the Windows-specific URL manipulation code to the new place where the URL is read from the command-line arguments. --- Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #833: MINIFICPP-1272 - Graceful shutdown on flow update
adamdebreceni commented on a change in pull request #833:
URL: https://github.com/apache/nifi-minifi-cpp/pull/833#discussion_r454243881
##
File path: libminifi/test/flow-tests/FlowControllerTests.cpp
##
@@ -119,7 +107,125 @@ TEST_CASE("Flow shutdown drains connections",
"[TestFlow1]") {
controller->stop(true);
+ REQUIRE(sinkProc->trigger_count == 0);
+
for (auto& it : connectionMap) {
REQUIRE(it.second->isEmpty());
}
}
+
+TEST_CASE("Flow shutdown waits for a while", "[TestFlow2]") {
+ TestControllerWithFlow testController(yamlConfig);
+ auto controller = testController.controller_;
+ auto root = testController.root_;
+
+ auto sourceProc =
std::static_pointer_cast(root->findProcessor("Generator"));
+ auto sinkProc =
std::static_pointer_cast(root->findProcessor("TestProcessor"));
+
+ // prevent the initial trigger
+ // in case the source got triggered
+ // and the scheduler triggers the sink
+ // before we could initiate the shutdown
+ sinkProc->yield(100);
+
+ testController.startFlow();
+
+ // wait for the source processor to enqueue its flowFiles
+ std::this_thread::sleep_for(std::chrono::milliseconds{50});
+
+ REQUIRE(sourceProc->trigger_count.load() == 1);
+ REQUIRE(sinkProc->trigger_count.load() == 0);
+
+ controller->stop(true);
+
+ REQUIRE(sourceProc->trigger_count.load() == 1);
+ REQUIRE(sinkProc->trigger_count.load() == 3);
+}
+
+TEST_CASE("Flow stopped after grace period", "[TestFlow3]") {
+ TestControllerWithFlow testController(yamlConfig);
+ auto controller = testController.controller_;
+ auto root = testController.root_;
+
+
testController.configuration_->set(minifi::Configure::nifi_flowcontroller_drain_timeout,
"1000 ms");
+
+ auto sourceProc =
std::static_pointer_cast(root->findProcessor("Generator"));
+ auto sinkProc =
std::static_pointer_cast(root->findProcessor("TestProcessor"));
+
+ // prevent the initial trigger
+ // in case the source got triggered
+ // and the scheduler triggers the sink
+ sinkProc->yield(100);
+
+ sinkProc->onTriggerCb_ = [&]{
+static std::atomic first_onTrigger{true};
+bool isFirst = true;
+// sleep only on the first trigger
+if (first_onTrigger.compare_exchange_strong(isFirst, false)) {
+ std::this_thread::sleep_for(std::chrono::milliseconds{1500});
+}
+ };
+
+ testController.startFlow();
+
+ // wait for the source processor to enqueue its flowFiles
+ std::this_thread::sleep_for(std::chrono::milliseconds{50});
Review comment:
`active_tasks_` is a strange animal, should be revisited, but waiting
for the FlowFiles to be enqueued into the connections through the
`ProcessGroup::getTotalFlowFileCount` method is a viable solution
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Resolved] (MINIFICPP-1213) ConfigurableComponent log too verbose
[
https://issues.apache.org/jira/browse/MINIFICPP-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Szasz resolved MINIFICPP-1213.
-
Resolution: Fixed
Fixed in context of MINIFICPP-1231 by [~adebreceni], see:
https://github.com/apache/nifi-minifi-cpp/pull/797#discussion_r446101100
> ConfigurableComponent log too verbose
> -
>
> Key: MINIFICPP-1213
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1213
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Dániel Bakai
>Priority: Blocker
> Fix For: 0.8.0
>
>
> If you configure secure C2 by setting the following in minifi.properties:
> {noformat}
> nifi.security.client.certificate=${MINIFI_HOME}/conf/client1_cert.pem
> nifi.security.client.private.key=${MINIFI_HOME}/conf/client1_key.pem
> #nifi.security.client.pass.phrase=
> nifi.security.client.ca.certificate=${MINIFI_HOME}/conf/client1_ca.pem{noformat}
> On every heartbeat a warning level log will be logged because the passphrase
> is empty:
> {noformat}
> [2020-04-30 04:18:32.921]
> [org::apache::nifi::minifi::core::ConfigurableComponent] [warning] Component
> Passphrase property name Passphrase, empty value
> [2020-04-30 04:18:33.922]
> [org::apache::nifi::minifi::core::ConfigurableComponent] [warning] Component
> Passphrase property name Passphrase, empty value
> [2020-04-30 04:18:34.924]
> [org::apache::nifi::minifi::core::ConfigurableComponent] [warning] Component
> Passphrase property name Passphrase, empty value
> [2020-04-30 04:18:35.925]
> [org::apache::nifi::minifi::core::ConfigurableComponent] [warning] Component
> Passphrase property name Passphrase, empty value
> [2020-04-30 04:18:36.922]
> [org::apache::nifi::minifi::core::ConfigurableComponent] [warning] Component
> Passphrase property name Passphrase, empty value
> [2020-04-30 04:18:37.921]
> [org::apache::nifi::minifi::core::ConfigurableComponent] [warning] Component
> Passphrase property name Passphrase, empty value
> [2020-04-30 04:18:38.920]
> [org::apache::nifi::minifi::core::ConfigurableComponent] [warning] Component
> Passphrase property name Passphrase, empty value
> [2020-04-30 04:18:39.921]
> [org::apache::nifi::minifi::core::ConfigurableComponent] [warning] Component
> Passphrase property name Passphrase, empty value
> [2020-04-30 04:18:40.923]
> [org::apache::nifi::minifi::core::ConfigurableComponent] [warning] Component
> Passphrase property name Passphrase, empty value
> [2020-04-30 04:18:41.922]
> [org::apache::nifi::minifi::core::ConfigurableComponent] [warning] Component
> Passphrase property name Passphrase, empty value
> [2020-04-30 04:18:42.925]
> [org::apache::nifi::minifi::core::ConfigurableComponent] [warning] Component
> Passphrase property name Passphrase, empty value
> [2020-04-30 04:18:43.923]
> [org::apache::nifi::minifi::core::ConfigurableComponent] [warning] Component
> Passphrase property name Passphrase, empty value
> [2020-04-30 04:18:44.921]
> [org::apache::nifi::minifi::core::ConfigurableComponent] [warning] Component
> Passphrase property name Passphrase, empty value
> [2020-04-30 04:18:45.925]
> [org::apache::nifi::minifi::core::ConfigurableComponent] [warning] Component
> Passphrase property name Passphrase, empty value
> [2020-04-30 04:18:46.921]
> [org::apache::nifi::minifi::core::ConfigurableComponent] [warning] Component
> Passphrase property name Passphrase, empty value
> [2020-04-30 04:18:47.923]
> [org::apache::nifi::minifi::core::ConfigurableComponent] [warning] Component
> Passphrase property name Passphrase, empty value
> [2020-04-30 04:18:48.923]
> [org::apache::nifi::minifi::core::ConfigurableComponent] [warning] Component
> Passphrase property name Passphrase, empty value
> [2020-04-30 04:18:49.924]
> [org::apache::nifi::minifi::core::ConfigurableComponent] [warning] Component
> Passphrase property name Passphrase, empty value
> [2020-04-30 04:18:50.920]
> [org::apache::nifi::minifi::core::ConfigurableComponent] [warning] Component
> Passphrase property name Passphrase, empty value
> [2020-04-30 04:18:51.920]
> [org::apache::nifi::minifi::core::ConfigurableComponent] [warning] Component
> Passphrase property name Passphrase, empty value{noformat}
>
> This is a completely valid case, so this log is too verbose, it should at max
> be debug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor
hunyadi-dev commented on a change in pull request #821:
URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r454235239
##
File path: extensions/standard-processors/processors/RetryFlowFile.cpp
##
@@ -0,0 +1,183 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "RetryFlowFile.h"
+
+#include "core/PropertyValidation.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property
RetryFlowFile::RetryAttribute(core::PropertyBuilder::createProperty("Retry
Attribute")
+->withDescription(
+"The name of the attribute that contains the current retry count for
the FlowFile."
+"WARNING: If the name matches an attribute already on the FlowFile
that does not contain a numerical value, "
+"the processor will either overwrite that attribute with '1' or fail
based on configuration.")
+->withDefaultValue("flowfile.retries")
+->supportsExpressionLanguage(true)
+->build());
+
+core::Property
RetryFlowFile::MaximumRetries(core::PropertyBuilder::createProperty("Maximum
Retries")
+->withDescription("The maximum number of times a FlowFile can be retried
before being passed to the 'retries_exceeded' relationship.")
+->withDefaultValue(3)
+->supportsExpressionLanguage(true)
+->build());
+
+core::Property
RetryFlowFile::PenalizeRetries(core::PropertyBuilder::createProperty("Penalize
Retries")
+ ->withDescription("If set to 'true', this Processor will penalize input
FlowFiles before passing them to the 'retry' relationship. This does not apply
to the 'retries_exceeded' relationship.")
+ ->withDefaultValue(true)
+ ->build());
+
+core::Property
RetryFlowFile::FailOnNonNumericalOverwrite(core::PropertyBuilder::createProperty("Fail
on Non-numerical Overwrite")
+->withDescription("If the FlowFile already has the attribute defined in
'Retry Attribute' that is *not* a number, fail the FlowFile instead of
resetting that value to '1'")
+->withDefaultValue(false)
+->build());
+
+core::Property
RetryFlowFile::ReuseMode(core::PropertyBuilder::createProperty("Reuse Mode")
+->withDescription(
+"Defines how the Processor behaves if the retry FlowFile has a
different retry UUID than "
+"the instance that received the FlowFile. This generally means that
the attribute was "
+"not reset after being successfully retried by a previous instance of
this processor.")
+->withAllowableValues({FAIL_ON_REUSE, WARN_ON_REUSE,
RESET_REUSE})
+->withDefaultValue(FAIL_ON_REUSE)
+->build());
+
+core::Relationship RetryFlowFile::Retry("retry",
+ "Input FlowFile has not exceeded the configured maximum retry count, pass
this relationship back to the input Processor to create a limited feedback
loop.");
+core::Relationship RetryFlowFile::RetriesExceeded("retries_exceeded",
+ "Input FlowFile has exceeded the configured maximum retry count, do not pass
this relationship back to the input Processor to terminate the limited feedback
loop.");
+core::Relationship RetryFlowFile::Failure("failure",
+"The processor is configured such that a non-numerical value on 'Retry
Attribute' results in a failure instead of resetting "
+"that value to '1'. This will immediately terminate the limited feedback
loop. Might also include when 'Maximum Retries' contains "
+" attribute expression language that does not resolve to an Integer.");
+
+void RetryFlowFile::initialize() {
+ setSupportedProperties({
+RetryAttribute,
+MaximumRetries,
+PenalizeRetries,
+FailOnNonNumericalOverwrite,
+ReuseMode,
+ });
+ setSupportedRelationships({
+Retry,
+RetriesExceeded,
+Failure,
+ });
+}
+
+void RetryFlowFile::onSchedule(core::ProcessContext* context,
core::ProcessSessionFactory* /* sessionFactory */) {
+ context->getProperty(RetryAttribute.getName(), retry_attribute_);
+ context->getProperty(MaximumRetries.getName(), maximum_retries_);
+ context->getProperty(PenalizeRetries.getName(), penalize_retries_);
+ context->getProperty(FailOnNonNumericalOverwrite.getName(),
fail_on_non_numerical_overwrite_);
+ context->getProperty(ReuseMode.getName(),
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor
adamdebreceni commented on a change in pull request #821:
URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r454193328
##
File path: extensions/standard-processors/processors/RetryFlowFile.cpp
##
@@ -0,0 +1,183 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "RetryFlowFile.h"
+
+#include "core/PropertyValidation.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property
RetryFlowFile::RetryAttribute(core::PropertyBuilder::createProperty("Retry
Attribute")
+->withDescription(
+"The name of the attribute that contains the current retry count for
the FlowFile."
+"WARNING: If the name matches an attribute already on the FlowFile
that does not contain a numerical value, "
+"the processor will either overwrite that attribute with '1' or fail
based on configuration.")
+->withDefaultValue("flowfile.retries")
+->supportsExpressionLanguage(true)
+->build());
+
+core::Property
RetryFlowFile::MaximumRetries(core::PropertyBuilder::createProperty("Maximum
Retries")
+->withDescription("The maximum number of times a FlowFile can be retried
before being passed to the 'retries_exceeded' relationship.")
+->withDefaultValue(3)
+->supportsExpressionLanguage(true)
+->build());
+
+core::Property
RetryFlowFile::PenalizeRetries(core::PropertyBuilder::createProperty("Penalize
Retries")
+ ->withDescription("If set to 'true', this Processor will penalize input
FlowFiles before passing them to the 'retry' relationship. This does not apply
to the 'retries_exceeded' relationship.")
+ ->withDefaultValue(true)
+ ->build());
+
+core::Property
RetryFlowFile::FailOnNonNumericalOverwrite(core::PropertyBuilder::createProperty("Fail
on Non-numerical Overwrite")
+->withDescription("If the FlowFile already has the attribute defined in
'Retry Attribute' that is *not* a number, fail the FlowFile instead of
resetting that value to '1'")
+->withDefaultValue(false)
+->build());
+
+core::Property
RetryFlowFile::ReuseMode(core::PropertyBuilder::createProperty("Reuse Mode")
+->withDescription(
+"Defines how the Processor behaves if the retry FlowFile has a
different retry UUID than "
+"the instance that received the FlowFile. This generally means that
the attribute was "
+"not reset after being successfully retried by a previous instance of
this processor.")
+->withAllowableValues({FAIL_ON_REUSE, WARN_ON_REUSE,
RESET_REUSE})
+->withDefaultValue(FAIL_ON_REUSE)
+->build());
+
+core::Relationship RetryFlowFile::Retry("retry",
+ "Input FlowFile has not exceeded the configured maximum retry count, pass
this relationship back to the input Processor to create a limited feedback
loop.");
+core::Relationship RetryFlowFile::RetriesExceeded("retries_exceeded",
+ "Input FlowFile has exceeded the configured maximum retry count, do not pass
this relationship back to the input Processor to terminate the limited feedback
loop.");
+core::Relationship RetryFlowFile::Failure("failure",
+"The processor is configured such that a non-numerical value on 'Retry
Attribute' results in a failure instead of resetting "
+"that value to '1'. This will immediately terminate the limited feedback
loop. Might also include when 'Maximum Retries' contains "
+" attribute expression language that does not resolve to an Integer.");
+
+void RetryFlowFile::initialize() {
+ setSupportedProperties({
+RetryAttribute,
+MaximumRetries,
+PenalizeRetries,
+FailOnNonNumericalOverwrite,
+ReuseMode,
+ });
+ setSupportedRelationships({
+Retry,
+RetriesExceeded,
+Failure,
+ });
+}
+
+void RetryFlowFile::onSchedule(core::ProcessContext* context,
core::ProcessSessionFactory* /* sessionFactory */) {
+ context->getProperty(RetryAttribute.getName(), retry_attribute_);
+ context->getProperty(MaximumRetries.getName(), maximum_retries_);
+ context->getProperty(PenalizeRetries.getName(), penalize_retries_);
+ context->getProperty(FailOnNonNumericalOverwrite.getName(),
fail_on_non_numerical_overwrite_);
+ context->getProperty(ReuseMode.getName(),
[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #836: MINIFICPP-1248 Create unit tests for the ConsumeWindowsEventLog processor
fgerlits commented on a change in pull request #836:
URL: https://github.com/apache/nifi-minifi-cpp/pull/836#discussion_r454180291
##
File path: extensions/windows-event-log/tests/ConsumeWindowsEventLogTests.cpp
##
@@ -0,0 +1,393 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "ConsumeWindowsEventLog.h"
+
+#include "core/ConfigurableComponent.h"
+#include "../../../../extensions/standard-processors/processors/LogAttribute.h"
+#include "TestBase.h"
+
+using ConsumeWindowsEventLog =
org::apache::nifi::minifi::processors::ConsumeWindowsEventLog;
+using LogAttribute = org::apache::nifi::minifi::processors::LogAttribute;
+using ConfigurableComponent =
org::apache::nifi::minifi::core::ConfigurableComponent;
+using IdGenerator = org::apache::nifi::minifi::utils::IdGenerator;
+
+namespace {
+
+core::Relationship Success{"success", "Everything is fine"};
+
+const std::string APPLICATION_CHANNEL = "Application";
+
+constexpr DWORD CWEL_TESTS_OPCODE = 14985; // random opcode hopefully won't
clash with something important
+
+void reportEvent(const std::string& channel, const char* message, WORD
log_level = EVENTLOG_INFORMATION_TYPE) {
+ auto event_source = RegisterEventSourceA(nullptr, channel.c_str());
+ auto deleter = gsl::finally([_source](){
DeregisterEventSource(event_source); });
+ ReportEventA(event_source, log_level, 0, CWEL_TESTS_OPCODE, nullptr, 1, 0,
, nullptr);
+}
+
+} // namespace
+
+TEST_CASE("ConsumeWindowsEventLog constructor works", "[create]") {
+ TestController test_controller;
+ std::shared_ptr test_plan = test_controller.createPlan();
+
+ REQUIRE_NOTHROW(ConsumeWindowsEventLog processor_one("one"));
+
+ REQUIRE_NOTHROW(
+utils::Identifier uuid = utils::IdGenerator::getIdGenerator()->generate();
+ConsumeWindowsEventLog processor_two("two", uuid);
+ ); // NOLINT
+
+ REQUIRE_NOTHROW(
+auto processor = test_plan->addProcessor("ConsumeWindowsEventLog", "cwel");
+ ); // NOLINT
+}
+
+TEST_CASE("ConsumeWindowsEventLog properties work with default values",
"[create][properties]") {
+ TestController test_controller;
+ LogTestController::getInstance().setDebug();
+ LogTestController::getInstance().setTrace();
+ std::shared_ptr test_plan = test_controller.createPlan();
+
+ auto processor = test_plan->addProcessor("ConsumeWindowsEventLog", "cwel");
+ test_controller.runSession(test_plan);
+
+ auto properties_required_or_with_default_value = {
+ConsumeWindowsEventLog::Channel,
+ConsumeWindowsEventLog::Query,
+// ConsumeWindowsEventLog::RenderFormatXML, // FIXME(fgerlits): not
defined, does not exist in NiFi either; should be removed
+ConsumeWindowsEventLog::MaxBufferSize,
+// ConsumeWindowsEventLog::InactiveDurationToReconnect, //
FIXME(fgerlits): obsolete, see definition; should be removed
+ConsumeWindowsEventLog::IdentifierMatcher,
+ConsumeWindowsEventLog::IdentifierFunction,
+ConsumeWindowsEventLog::ResolveAsAttributes,
+ConsumeWindowsEventLog::EventHeader,
+ConsumeWindowsEventLog::OutputFormat,
+ConsumeWindowsEventLog::BatchCommitSize,
+ConsumeWindowsEventLog::BookmarkRootDirectory,
+ConsumeWindowsEventLog::ProcessOldEvents
+ };
+ for (const core::Property& property :
properties_required_or_with_default_value) {
+if (!LogTestController::getInstance().contains("property name " +
property.getName() + " value ")) {
+ FAIL("Property did not get queried: " << property.getName());
+}
+ }
+
+ auto properties_optional_without_default_value = {
+ConsumeWindowsEventLog::EventHeaderDelimiter
+ };
+ for (const core::Property& property :
properties_optional_without_default_value) {
+if (!LogTestController::getInstance().contains("property name " +
property.getName() + ", empty value")) {
+ FAIL("Optional property did not get queried: " << property.getName());
+}
+ }
+
+ REQUIRE(LogTestController::getInstance().contains("Successfully configured
CWEL"));
+}
+
+TEST_CASE("ConsumeWindowsEventLog onSchedule throws if it cannot create the
bookmark", "[create][bookmark]") {
+ TestController test_controller;
+ std::shared_ptr test_plan = test_controller.createPlan();
+
+ auto processor =
[jira] [Updated] (MINIFICPP-1292) Properties with an allowed list of values should check their input
[
https://issues.apache.org/jira/browse/MINIFICPP-1292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ferenc Gerlits updated MINIFICPP-1292:
--
Description:
If you declare a Property of a processor with a list of allowed values (using
{{withAllowableValues}}) and then give this property a non-allowed value when
you configure the processor, it will not automatically throw an exception, only
if the processor code checks the property in {{onSchedule()}}.
The heartbeat will contain the list of allowed values, so if you use a C2
client to set up the configuration, then this can be prevented by the client,
so this is not a major issue.
Nevertheless, the Property itself should verify that it is valid, so each
processor doesn't need to check them in {{onSchedule()}}, or risk not working
correctly when configured manually. Since this change may break existing
deployments, it should be rolled out with a major version change.
was:
If you declare a Property of a processor with a list of allowed values (using
{{withAllowableValues}}) and then give this property a non-allowed value when
you configure the processor, it will not automatically throw an exception, only
if the processor code checks the property in {{onSchedule()}}.
The heartbeat will contain the list of allowed values, so if you use a C2
client to set up the configuration, than this can be prevented by the client,
so this is not a major issue.
Nevertheless, the Property itself should verify that it is valid, so each
processor doesn't need to check them in {{onSchedule()}}, or risk not working
correctly when configured manually. Since this change may break existing
deployments, it should be rolled out with a major version change.
> Properties with an allowed list of values should check their input
> --
>
> Key: MINIFICPP-1292
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1292
> Project: Apache NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Ferenc Gerlits
>Priority: Minor
> Fix For: 1.0.0
>
>
> If you declare a Property of a processor with a list of allowed values (using
> {{withAllowableValues}}) and then give this property a non-allowed value when
> you configure the processor, it will not automatically throw an exception,
> only if the processor code checks the property in {{onSchedule()}}.
> The heartbeat will contain the list of allowed values, so if you use a C2
> client to set up the configuration, then this can be prevented by the client,
> so this is not a major issue.
> Nevertheless, the Property itself should verify that it is valid, so each
> processor doesn't need to check them in {{onSchedule()}}, or risk not working
> correctly when configured manually. Since this change may break existing
> deployments, it should be rolled out with a major version change.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (MINIFICPP-1292) Properties with an allowed list of values should check their input
Ferenc Gerlits created MINIFICPP-1292:
-
Summary: Properties with an allowed list of values should check
their input
Key: MINIFICPP-1292
URL: https://issues.apache.org/jira/browse/MINIFICPP-1292
Project: Apache NiFi MiNiFi C++
Issue Type: Improvement
Reporter: Ferenc Gerlits
Fix For: 1.0.0
If you declare a Property of a processor with a list of allowed values (using
{{withAllowableValues}}) and then give this property a non-allowed value when
you configure the processor, it will not automatically throw an exception, only
if the processor code checks the property in {{onSchedule()}}.
The heartbeat will contain the list of allowed values, so if you use a C2
client to set up the configuration, than this can be prevented by the client,
so this is not a major issue.
Nevertheless, the Property itself should verify that it is valid, so each
processor doesn't need to check them in {{onSchedule()}}, or risk not working
correctly when configured manually. Since this change may break existing
deployments, it should be rolled out with a major version change.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] hunyadi-dev opened a new pull request #837: MINIFICPP-1121 - Upgrade spdlog to version 1.7.0
hunyadi-dev opened a new pull request #837: URL: https://github.com/apache/nifi-minifi-cpp/pull/837 This is a draft PR used for running CI jobs on different platforms. Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] s9514171 opened a new pull request #4407: NIFI-7623 Take unnecessary behavior out when using registry to update…
s9514171 opened a new pull request #4407:
URL: https://github.com/apache/nifi/pull/4407
… the flow
Description of PR
When we're using registry to update the flow, there are some unnecessary
behavior stop and start processors.
such as
processor: color style, position changes
connection: bend points changed
This PR is used to remove these kind of behaviors
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `master`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #833: MINIFICPP-1272 - Graceful shutdown on flow update
adamdebreceni commented on a change in pull request #833:
URL: https://github.com/apache/nifi-minifi-cpp/pull/833#discussion_r454137073
##
File path: libminifi/src/FlowController.cpp
##
@@ -239,8 +239,35 @@ int16_t FlowController::stop(bool force, uint64_t
timeToWait) {
if (running_) {
// immediately indicate that we are not running
logger_->log_info("Stop Flow Controller");
-if (this->root_)
+if (this->root_) {
+ // stop source processors first
+ this->root_->stopProcessing(timer_scheduler_, event_scheduler_,
cron_scheduler_, [] (const std::shared_ptr& proc) -> bool {
+return !proc->hasIncomingConnections();
+ });
+ std::chrono::milliseconds shutdown_timer{0};
+ // we enable C2 to progressively increase the timeout
+ // in case it sees that waiting for a little longer could
+ // allow the FlowFiles to be processed
+ auto shutdown_timeout = [&]() -> std::chrono::milliseconds {
+if (timeToWait != 0) {
+ return std::chrono::milliseconds{timeToWait};
+}
+static const core::TimePeriodValue default_timeout{"10 sec"};
+utils::optional shutdown_timeout;
+std::string shutdown_timeout_str;
+if
(configuration_->get(minifi::Configure::nifi_flowcontroller_drain_timeout,
shutdown_timeout_str)) {
+ shutdown_timeout =
core::TimePeriodValue::fromString(shutdown_timeout_str);
+}
+return
std::chrono::milliseconds{shutdown_timeout.value_or(default_timeout).getMilliseconds()};
+ };
+ std::size_t count;
+ while (shutdown_timer < shutdown_timeout() && (count =
this->root_->getTotalFlowFileCount()) != 0) {
+std::this_thread::sleep_for(shutdown_check_interval_);
+shutdown_timer += shutdown_check_interval_;
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] s9514171 closed pull request #4406: NIFI-7623 Take unnecessary behavior out when using registry to update…
s9514171 closed pull request #4406: URL: https://github.com/apache/nifi/pull/4406 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
