[GitHub] [nifi] CDann3r commented on issue #3956: Added NTLM Authentication based on presence of NTLM Domain property
CDann3r commented on issue #3956: Added NTLM Authentication based on presence of NTLM Domain property URL: https://github.com/apache/nifi/pull/3956#issuecomment-579521893 I think the issue is that the used httpClient (okhttp3.OkHttpClient) doesn't support NTLM. Therefore two complete java classes have been added which handle the actual NTLM. I propose to use "org.apache.http.impl.client.HttpClientBuilder" as @simonellistonball did with his WebDAV Processor which is Licensed under Apache2. Reference => see addAuth method which creates a CredentialProvider for the HttpClientBuilder [AbstractWebDAVProcessor.java](https://github.com/simonellistonball/nifi-webdav-bundle/blob/master/nifi-webdav-processors/src/main/java/org/apache/nifi/processors/webdav/AbstractWebDAVProcessor.java) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (NIFI-6721) jms_expiration attribute problem
[ https://issues.apache.org/jira/browse/NIFI-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025476#comment-17025476 ] Joey Frazee edited comment on NIFI-6721 at 1/28/20 10:32 PM: - Looks like the source of the mistake is that the Spring JmsTemplate doesn't actually have a setExpiration() so as you're seeing, the value got used with setTimeToLive(): https://github.com/apache/nifi/blob/rel/nifi-1.11.0/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L85 was (Author: jfrazee): Looks like the source of the mistake is that the Spring JmsTemplate doesn't actually have a setExpiration() so as you're seeing, the value got used with setTimeToLive(): https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L85 > jms_expiration attribute problem > > > Key: NIFI-6721 > URL: https://issues.apache.org/jira/browse/NIFI-6721 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.8.0 > Environment: Linux CENTOS 7 >Reporter: Tim Chermak >Priority: Minor > > The documentation for PublishJMS indicates the JMSExpiration is set with the > attribute jms_expiration. However, this value is really the time-to-live > (ttl) in milliseconds. The JMSExpiration is calculated by the provider > library as "expiration = timestamp + ttl" > So, this NiFi flowfile attribute should really be named jms_ttl. The current > setup works correctly when NiFi creates and publishes a message, but has > problems when you try to republish a JMS message. > GetFile -> UpdateAttibute -> PublishJMS creates a valid JMSExpiration in the > message, however, when a JMS has the expiration set, ConsumeJMS -> PublishJMS > shows an error in the nifi.--app.log file: > "o.apache.nifi.jms.processors.PublishJMS PublishJMS[id=016b1005-xx...] > Incompatible value for attribute jms_expiration [1566428032803] is not a > number. Ignoring this attribute." > Looks like ConsumeJMS set the flowfile attribute to the expiration value > rather than the time-ti-live value. Time-to-live should be jms_ttl = > expiration - current_time. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6721) jms_expiration attribute problem
[ https://issues.apache.org/jira/browse/NIFI-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025476#comment-17025476 ] Joey Frazee commented on NIFI-6721: --- Looks like the source of the mistake is that the Spring JmsTemplate doesn't actually have a setExpiration() so as you're seeing, the value got used with setTimeToLive(): https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L85 > jms_expiration attribute problem > > > Key: NIFI-6721 > URL: https://issues.apache.org/jira/browse/NIFI-6721 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.8.0 > Environment: Linux CENTOS 7 >Reporter: Tim Chermak >Priority: Minor > > The documentation for PublishJMS indicates the JMSExpiration is set with the > attribute jms_expiration. However, this value is really the time-to-live > (ttl) in milliseconds. The JMSExpiration is calculated by the provider > library as "expiration = timestamp + ttl" > So, this NiFi flowfile attribute should really be named jms_ttl. The current > setup works correctly when NiFi creates and publishes a message, but has > problems when you try to republish a JMS message. > GetFile -> UpdateAttibute -> PublishJMS creates a valid JMSExpiration in the > message, however, when a JMS has the expiration set, ConsumeJMS -> PublishJMS > shows an error in the nifi.--app.log file: > "o.apache.nifi.jms.processors.PublishJMS PublishJMS[id=016b1005-xx...] > Incompatible value for attribute jms_expiration [1566428032803] is not a > number. Ignoring this attribute." > Looks like ConsumeJMS set the flowfile attribute to the expiration value > rather than the time-ti-live value. Time-to-live should be jms_ttl = > expiration - current_time. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6721) jms_expiration attribute problem
[ https://issues.apache.org/jira/browse/NIFI-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025468#comment-17025468 ] Joey Frazee commented on NIFI-6721: --- [~tchermak] what provider(s) have you seen this with? > jms_expiration attribute problem > > > Key: NIFI-6721 > URL: https://issues.apache.org/jira/browse/NIFI-6721 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.8.0 > Environment: Linux CENTOS 7 >Reporter: Tim Chermak >Priority: Minor > > The documentation for PublishJMS indicates the JMSExpiration is set with the > attribute jms_expiration. However, this value is really the time-to-live > (ttl) in milliseconds. The JMSExpiration is calculated by the provider > library as "expiration = timestamp + ttl" > So, this NiFi flowfile attribute should really be named jms_ttl. The current > setup works correctly when NiFi creates and publishes a message, but has > problems when you try to republish a JMS message. > GetFile -> UpdateAttibute -> PublishJMS creates a valid JMSExpiration in the > message, however, when a JMS has the expiration set, ConsumeJMS -> PublishJMS > shows an error in the nifi.--app.log file: > "o.apache.nifi.jms.processors.PublishJMS PublishJMS[id=016b1005-xx...] > Incompatible value for attribute jms_expiration [1566428032803] is not a > number. Ignoring this attribute." > Looks like ConsumeJMS set the flowfile attribute to the expiration value > rather than the time-ti-live value. Time-to-live should be jms_ttl = > expiration - current_time. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6780) Create a Metrics Query Reporting Task
[ https://issues.apache.org/jira/browse/NIFI-6780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025431#comment-17025431 ] Behrouz commented on NIFI-6780: --- Could you please give us an example how to use this service. It would be better if you write an article with some examples. > Create a Metrics Query Reporting Task > - > > Key: NIFI-6780 > URL: https://issues.apache.org/jira/browse/NIFI-6780 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Yolanda M. Davis >Assignee: Matt Burgess >Priority: Major > Fix For: 1.10.0 > > Time Spent: 2.5h > Remaining Estimate: 0h > > Currently NiFi has metrics reporting tasks which have a specific set of > metrics that are sent out to via site-to-site or other protocols. To expand > upon this a query based reporting task is proposed to provide users the > flexibility to select the types of metrics and the conditions on they should > be reported using sql like statements. > It may be desired that the results of a query are transmitted to any number > of targets. The current pattern is to implement this as a Site-to-Site > reporting task, but that puts the onus on the user to create a sub-flow with > an Input Port for receiving the S2S messages, and it creates additional > provenance events for these. A new approach to be considered here is to > decouple the results from the destination. Proposed is a RecordSinkService > controller service interface, which the query-based reporting task uses to > transmit the query results. The configured RecordSinkService implementation > would be responsible for the actual transmission of results to the sink. > Possible initial implementations include a Site-To-Site RecordSink (for > feature parity with the other reporting tasks) and a DatabaseRecordSink (to > transmit the query results to an external RDBMS using DBCPConnectionPool). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7073) PutHDFS silently ignores exceptions when closing temporary file output stream
[
https://issues.apache.org/jira/browse/NIFI-7073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Hogue updated NIFI-7073:
Description:
The PutHDFS processor can silently fail when closing an output stream when it's
finished writing FlowFiles to a temporary space in HDFS. {{DFSOutputStream}}
can throw an {{IOException}} when it's closed and that's ignored by the
processor. This results in the FlowFile getting dropped (e.g. a loss of data)
when it should be routed to the failure relationship.
Source:
[https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/PutHDFS.java#L365]
Instead, the exception should be caught & thrown and the FlowFile should be
routed to the failure relationship. There's probably an argument to avoid
ignoring _any_ exception.
was:
The PutHDFS processor can silently fail when closing an output stream when it's
finished writing FlowFiles to a temporary space in HDFS. {{DFSOutputStream}}
can throw an {{IOException}} when it's closed and that's ignored by the
processor. This results in the FlowFile getting dropped (e.g. a loss of data)
when it should be routed to the failure relationship.
Source:
[https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/PutHDFS.java#L365]
Instead, the exception should be caught & thrown and the FlowFile should be
routed to the failure relationship.
> PutHDFS silently ignores exceptions when closing temporary file output stream
> -
>
> Key: NIFI-7073
> URL: https://issues.apache.org/jira/browse/NIFI-7073
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.11.0
>Reporter: Michael Hogue
>Priority: Major
>
> The PutHDFS processor can silently fail when closing an output stream when
> it's finished writing FlowFiles to a temporary space in HDFS.
> {{DFSOutputStream}} can throw an {{IOException}} when it's closed and that's
> ignored by the processor. This results in the FlowFile getting dropped (e.g.
> a loss of data) when it should be routed to the failure relationship.
> Source:
> [https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/PutHDFS.java#L365]
> Instead, the exception should be caught & thrown and the FlowFile should be
> routed to the failure relationship. There's probably an argument to avoid
> ignoring _any_ exception.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-7073) PutHDFS silently ignores exceptions when closing temporary file output stream
[
https://issues.apache.org/jira/browse/NIFI-7073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Hogue updated NIFI-7073:
Summary: PutHDFS silently ignores exceptions when closing temporary file
output stream (was: PutHDFS silently ignores exceptions when writing temporary
file)
> PutHDFS silently ignores exceptions when closing temporary file output stream
> -
>
> Key: NIFI-7073
> URL: https://issues.apache.org/jira/browse/NIFI-7073
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.11.0
>Reporter: Michael Hogue
>Priority: Major
>
> The PutHDFS processor can silently fail when closing an output stream when
> it's finished writing FlowFiles to a temporary space in HDFS.
> {{DFSOutputStream}} can throw an {{IOException}} when it's closed and that's
> ignored by the processor. This results in the FlowFile getting dropped (e.g.
> a loss of data) when it should be routed to the failure relationship.
>
> Instead, the exception should be caught & thrown and the FlowFile should be
> routed to the failure relationship.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-7073) PutHDFS silently ignores exceptions when closing temporary file output stream
[
https://issues.apache.org/jira/browse/NIFI-7073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Hogue updated NIFI-7073:
Description:
The PutHDFS processor can silently fail when closing an output stream when it's
finished writing FlowFiles to a temporary space in HDFS. {{DFSOutputStream}}
can throw an {{IOException}} when it's closed and that's ignored by the
processor. This results in the FlowFile getting dropped (e.g. a loss of data)
when it should be routed to the failure relationship.
Source:
[https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/PutHDFS.java#L365]
Instead, the exception should be caught & thrown and the FlowFile should be
routed to the failure relationship.
was:
The PutHDFS processor can silently fail when closing an output stream when it's
finished writing FlowFiles to a temporary space in HDFS. {{DFSOutputStream}}
can throw an {{IOException}} when it's closed and that's ignored by the
processor. This results in the FlowFile getting dropped (e.g. a loss of data)
when it should be routed to the failure relationship.
Instead, the exception should be caught & thrown and the FlowFile should be
routed to the failure relationship.
> PutHDFS silently ignores exceptions when closing temporary file output stream
> -
>
> Key: NIFI-7073
> URL: https://issues.apache.org/jira/browse/NIFI-7073
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.11.0
>Reporter: Michael Hogue
>Priority: Major
>
> The PutHDFS processor can silently fail when closing an output stream when
> it's finished writing FlowFiles to a temporary space in HDFS.
> {{DFSOutputStream}} can throw an {{IOException}} when it's closed and that's
> ignored by the processor. This results in the FlowFile getting dropped (e.g.
> a loss of data) when it should be routed to the failure relationship.
> Source:
> [https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/PutHDFS.java#L365]
> Instead, the exception should be caught & thrown and the FlowFile should be
> routed to the failure relationship.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (NIFI-7073) PutHDFS silently ignores exceptions when writing temporary file
Michael Hogue created NIFI-7073:
---
Summary: PutHDFS silently ignores exceptions when writing
temporary file
Key: NIFI-7073
URL: https://issues.apache.org/jira/browse/NIFI-7073
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Affects Versions: 1.11.0
Reporter: Michael Hogue
The PutHDFS processor can silently fail when closing an output stream when it's
finished writing FlowFiles to a temporary space in HDFS. {{DFSOutputStream}}
can throw an {{IOException}} when it's closed and that's ignored by the
processor. This results in the FlowFile getting dropped (e.g. a loss of data)
when it should be routed to the failure relationship.
Instead, the exception should be caught & thrown and the FlowFile should be
routed to the failure relationship.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (NIFI-7072) ForkRecord in Extract mode fails if concurrency is increased.
Branko Peshevski created NIFI-7072:
--
Summary: ForkRecord in Extract mode fails if concurrency is
increased.
Key: NIFI-7072
URL: https://issues.apache.org/jira/browse/NIFI-7072
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Affects Versions: 1.11.0, 1.9.2, 1.10.0
Environment: Java 8, Macosx,
Reporter: Branko Peshevski
Attachments: ForkRecord_concurrency_bug.xml
I have followed the example for ForkRecord from [additional details
page|https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.9.2/org.apache.nifi.processors.standard.ForkRecord/additionalDetails.html]
for multi-nested arrays and discovered that if the concurrency is increased
the processor and the content repository fails.
{code:java}
2020-01-28 18:59:07,485 ERROR [Timer-Driven Process Thread-5]
o.a.n.c.r.StandardProcessSession Failed to write content to
StandardFlowFileRecord[uuid=d55d9aa0-6b0b-4f6d-aca6-b340467cf29e,claim=,offset=0,name=f15480b3-57da-4e3e-9f5d-5cf2839bc2bd,size=0];
rolling back session java.io.IOException: Stream is closed at
org.apache.nifi.controller.repository.FileSystemRepository$ContentRepositoryOutputStream.write(FileSystemRepository.java:1855)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:122) at
org.apache.nifi.controller.repository.claim.ContentClaimWriteCache$1.write(ContentClaimWriteCache.java:101)
at
org.apache.nifi.controller.repository.io.DisableOnCloseOutputStream.write(DisableOnCloseOutputStream.java:49)
at
org.apache.nifi.stream.io.ByteCountingOutputStream.write(ByteCountingOutputStream.java:49)
at
org.apache.nifi.controller.repository.StandardProcessSession$7.write(StandardProcessSession.java:2636)
at
org.apache.nifi.controller.repository.io.TaskTerminationOutputStream.write(TaskTerminationOutputStream.java:62)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:122) at
org.apache.avro.io.BufferedBinaryEncoder$OutputStreamSink.innerWrite(BufferedBinaryEncoder.java:216)
at
org.apache.avro.io.BufferedBinaryEncoder.writeFixed(BufferedBinaryEncoder.java:150)
at
org.apache.avro.io.BlockingBinaryEncoder.flush(BlockingBinaryEncoder.java:244)
at
org.apache.nifi.avro.WriteAvroResultWithExternalSchema.flush(WriteAvroResultWithExternalSchema.java:94)
at
org.apache.nifi.avro.WriteAvroResultWithExternalSchema.onFinishRecordSet(WriteAvroResultWithExternalSchema.java:74)
at
org.apache.nifi.serialization.AbstractRecordSetWriter.finishRecordSet(AbstractRecordSetWriter.java:91)
at
org.apache.nifi.processors.standard.ForkRecord$1.process(ForkRecord.java:319)
at
org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2315)
at
org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2283)
at
org.apache.nifi.processors.standard.ForkRecord.onTrigger(ForkRecord.java:238)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1176)
at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) 2020-01-28 18:59:07,485 ERROR
[Timer-Driven Process Thread-5] o.a.nifi.processors.standard.ForkRecord
ForkRecord[id=4e463376-2d56-1d09-dbf6-1cf419144e8a] Failed to fork
StandardFlowFileRecord[uuid=82366dc4-e256-40c4-aec9-d5c681d0ba16,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1580234187165-76713,
container=default, section=937], offset=487914,
length=81319],offset=0,name=82366dc4-e256-40c4-aec9-d5c681d0ba16,size=81319]:
org.apache.nifi.processor.exception.FlowFileAccessException: Failed to write to
Content Repository for
StandardFlowFileRecord[uuid=d55d9aa0-6b0b-4f6d-aca6-b340467cf29e,claim=,offset=0,name=f15480b3-57da-4e3e-9f5d-5cf2839bc2bd,size=0]
org.apache.nifi.processor.exception.FlowFileAccessException: Failed to write
to Content Repository for
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371940861
##
File path: nanofi/src/coap/c2payload.c
##
@@ -0,0 +1,701 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+#include
+#include
+
+typedef enum {
+TYPE,
+NAME,
+INPUT,
+OUTPUT,
+IDENTIFIER,
+DISPLAYNAME,
+DESCRIPTION,
+VALIDATOR,
+SENSITIVE,
+DYNAMIC,
+REQUIRED,
+PROPERTIES,
+VERSION,
+STATUS,
+DEVICEINFO,
+SYSTEMFINO,
+MACHINEARCH,
+VCORES,
+PHYSICALMEMORYBYTES,
+NETWORKINFO,
+HOSTNAME,
+IPADDRESS,
+AGENTINFO,
+AGENTCLASS,
+UPTIME,
+AGENTMANIFEST,
+AGENTTYPE,
+IOMANIFEST,
+PROPERTYDESCRIPTORS,
+ECUINFO,
+OPERATION,
+OPERAND,
+FILEPATH,
+CHUNKSIZEBYTES,
+DELIMITER,
+TAILFREQMILLISECONDS,
+TCPPORT,
+NIFIPORT,
+REQUESTEDOPERATIONS
+} c2_keys_t;
+
+value_t value_uint8(uint8_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint8 = value;
+val.val_type = UINT8_TYPE;
+return val;
+}
+
+value_t value_uint16(uint16_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint16 = value;
+val.val_type = UINT16_TYPE;
+return val;
+}
+
+value_t value_uint32(uint32_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint32 = value;
+val.val_type = UINT32_TYPE;
+return val;
+}
+
+value_t value_uint64(uint64_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint64 = value;
+val.val_type = UINT64_TYPE;
+return val;
+}
+
+value_t value_string(const char * value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+copystr(value, _str);
+val.val_type = STRING_TYPE;
+return val;
+}
+
+value_t value_nstring(const unsigned char * value, size_t len) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_str = (char *)malloc(len + 1);
+memset(val.v_str, 0, len + 1);
+memcpy(val.v_str, value, len);
+val.val_type = STRING_TYPE;
+return val;
+}
+
+value_t value_map(c2_payload_map_t * value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_map = value;
+val.val_type = HASH_TYPE;
+return val;
+}
+
+value_t value_list(c2_payload_list_t * value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_maplist = value;
+val.val_type = LIST_TYPE;
+return val;
+}
+
+value_t value_property(properties_t * props) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_props = clone_properties(props);
+val.val_type = PROP_TYPE;
+return val;
+}
+
+value_t value_none() {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.val_type = NONE_TYPE;
+return val;
+}
+
+int is_value_none(value_t val) {
+return val.val_type == NONE_TYPE;
+}
+
+int is_value_uint(value_t val) {
+return val.val_type == UINT8_TYPE ||
+val.val_type == UINT16_TYPE ||
+val.val_type == UINT32_TYPE ||
+val.val_type == UINT64_TYPE;
+}
+
+int is_value_string(value_t val) {
+return val.val_type == STRING_TYPE;
+}
+
+int is_value_map(value_t val) {
+return val.val_type == HASH_TYPE;
+}
+
+int is_value_list(value_t val) {
+return val.val_type == LIST_TYPE;
+}
+
+int is_value_prop(value_t val) {
+return val.val_type == PROP_TYPE;
+}
+
+c2_payload_map_t * allocate_c2_payload() {
+c2_payload_map_t * map = (c2_payload_map_t
*)malloc(sizeof(c2_payload_map_t));
+memset(map, 0, sizeof(c2_payload_map_t));
+return map;
+}
+
+c2_payload_map_t * add_kvp(c2_payload_map_t ** map, uint16_t key, value_t
value) {
+c2_payload_map_t * field = allocate_c2_payload();
+field->key = key;
+field->value = value;
+HASH_ADD_INT((*map), key, field);
+return field;
+}
+
+c2_payload_list_t * add_list(c2_payload_list_t ** list, value_t value) {
+if (!list) return NULL;
+c2_payload_list_t * node = (c2_payload_list_t
*)malloc(sizeof(c2_payload_list_t));
+memset(node, 0, sizeof(c2_payload_list_t));
+node->value = value;
+LL_APPEND((*list), node);
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371963759
##
File path: nanofi/src/coap/c2payload.c
##
@@ -0,0 +1,701 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+#include
+#include
+
+typedef enum {
+TYPE,
+NAME,
+INPUT,
+OUTPUT,
+IDENTIFIER,
+DISPLAYNAME,
+DESCRIPTION,
+VALIDATOR,
+SENSITIVE,
+DYNAMIC,
+REQUIRED,
+PROPERTIES,
+VERSION,
+STATUS,
+DEVICEINFO,
+SYSTEMFINO,
+MACHINEARCH,
+VCORES,
+PHYSICALMEMORYBYTES,
+NETWORKINFO,
+HOSTNAME,
+IPADDRESS,
+AGENTINFO,
+AGENTCLASS,
+UPTIME,
+AGENTMANIFEST,
+AGENTTYPE,
+IOMANIFEST,
+PROPERTYDESCRIPTORS,
+ECUINFO,
+OPERATION,
+OPERAND,
+FILEPATH,
+CHUNKSIZEBYTES,
+DELIMITER,
+TAILFREQMILLISECONDS,
+TCPPORT,
+NIFIPORT,
+REQUESTEDOPERATIONS
+} c2_keys_t;
+
+value_t value_uint8(uint8_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint8 = value;
+val.val_type = UINT8_TYPE;
+return val;
+}
+
+value_t value_uint16(uint16_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint16 = value;
+val.val_type = UINT16_TYPE;
+return val;
+}
+
+value_t value_uint32(uint32_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint32 = value;
+val.val_type = UINT32_TYPE;
+return val;
+}
+
+value_t value_uint64(uint64_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint64 = value;
+val.val_type = UINT64_TYPE;
+return val;
+}
+
+value_t value_string(const char * value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+copystr(value, _str);
+val.val_type = STRING_TYPE;
+return val;
+}
+
+value_t value_nstring(const unsigned char * value, size_t len) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_str = (char *)malloc(len + 1);
+memset(val.v_str, 0, len + 1);
+memcpy(val.v_str, value, len);
+val.val_type = STRING_TYPE;
+return val;
+}
+
+value_t value_map(c2_payload_map_t * value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_map = value;
+val.val_type = HASH_TYPE;
+return val;
+}
+
+value_t value_list(c2_payload_list_t * value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_maplist = value;
+val.val_type = LIST_TYPE;
+return val;
+}
+
+value_t value_property(properties_t * props) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_props = clone_properties(props);
+val.val_type = PROP_TYPE;
+return val;
+}
+
+value_t value_none() {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.val_type = NONE_TYPE;
+return val;
+}
+
+int is_value_none(value_t val) {
+return val.val_type == NONE_TYPE;
+}
+
+int is_value_uint(value_t val) {
+return val.val_type == UINT8_TYPE ||
+val.val_type == UINT16_TYPE ||
+val.val_type == UINT32_TYPE ||
+val.val_type == UINT64_TYPE;
+}
+
+int is_value_string(value_t val) {
+return val.val_type == STRING_TYPE;
+}
+
+int is_value_map(value_t val) {
+return val.val_type == HASH_TYPE;
+}
+
+int is_value_list(value_t val) {
+return val.val_type == LIST_TYPE;
+}
+
+int is_value_prop(value_t val) {
+return val.val_type == PROP_TYPE;
+}
+
+c2_payload_map_t * allocate_c2_payload() {
+c2_payload_map_t * map = (c2_payload_map_t
*)malloc(sizeof(c2_payload_map_t));
+memset(map, 0, sizeof(c2_payload_map_t));
+return map;
+}
+
+c2_payload_map_t * add_kvp(c2_payload_map_t ** map, uint16_t key, value_t
value) {
+c2_payload_map_t * field = allocate_c2_payload();
+field->key = key;
+field->value = value;
+HASH_ADD_INT((*map), key, field);
+return field;
+}
+
+c2_payload_list_t * add_list(c2_payload_list_t ** list, value_t value) {
+if (!list) return NULL;
+c2_payload_list_t * node = (c2_payload_list_t
*)malloc(sizeof(c2_payload_list_t));
+memset(node, 0, sizeof(c2_payload_list_t));
+node->value = value;
+LL_APPEND((*list), node);
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371958945
##
File path: nanofi/src/core/string_utils.c
##
@@ -162,3 +163,87 @@ token_list tokenize_string_tailfile(const char * str,
char delim) {
tks.has_non_delimited_token = 0;
return tks;
}
+
+void copystr(const char * source, char ** dest) {
+if (!source || !dest) return;
+size_t len = strlen(source);
+char * tmp = (char *)malloc(len + 1);
+memset(tmp, 0, len + 1);
+memcpy(tmp, source, len);
+*dest = tmp;
+}
+
+void copynstr(const char * source, size_t len, char ** dest) {
+if (!source || !len || !dest) return;
+size_t source_len = strlen(source);
+char * tmp;
+if (source_len <= len) {
+copystr(source, );
+} else {
+tmp = (char *)malloc(len + 1);
+memcpy(tmp, source, len);
+tmp[len] = '\0';
+}
+*dest = tmp;
+}
+
+void copynstrd(const char * source, size_t src_len, unsigned char * dest,
size_t dest_len) {
+if (!source || !src_len || !dest || !dest_len) return;
+size_t min_len = MIN(src_len, dest_len);
+if (src_len == min_len && src_len != dest_len) {
+memcpy(dest, source, min_len);
+} else {
+ memcpy(dest, source, min_len - 1);
+}
+dest[min_len] = '\0';
+}
+
+
+int str_to_uint(const char * input_str, uint64_t * out) {
+if (!input_str) {
+return -1;
+}
+errno = 0;
+*out = (uint64_t)(strtoul(input_str, NULL, 10));
+if (errno != 0) {
+return -1;
+}
+return 0;
+}
+
+const char * uint_to_str(uint64_t value) {
+char value_str[21];
+snprintf(value_str, sizeof(value_str), "%llu", value);
Review comment:
The `"llu"` format specifier corresponds to `unsigned long long`. For
`uint64_t`, use `PRIu64` from `inttypes.h`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371778635
##
File path: nanofi/include/core/synchutils.h
##
@@ -0,0 +1,82 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef SYNCHUTILS_H_
+#define SYNCHUTILS_H_
+
+#ifdef WIN32
+#include
+#include
+#include
+#else
+#include
+#include
+#endif
+#include
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+typedef struct lock {
+#ifdef WIN32
+CRITICAL_SECTION cs;
+#else
+pthread_mutex_t mutex;
+#endif
+} lock_t ;
+
+typedef struct conditionvariable {
+#ifdef WIN32
+CONDITION_VARIABLE cv;
+#else
+pthread_cond_t cv;
+#endif
+} conditionvariable_t;
+
+typedef struct conditionvariable_attr {
+#ifndef WIN32
+pthread_condattr_t cv_attr;
+#endif
+unsigned int initialized : 1;
+} conditionvariable_attr_t;
Review comment:
Introducing platform-independent types for synchronization is a great idea,
but what's the reason for wrapping in a `struct` in addition to `typedef`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371959529
##
File path: nanofi/src/core/synchutils.c
##
@@ -0,0 +1,189 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+#include
+#include
+#include
+
+void initialize_lock(lock_t * lock) {
+#ifndef WIN32
+pthread_mutex_init(>mutex, NULL);
+#else
+InitializeCriticalSection(>cs);
+#endif
+}
+
+#ifndef WIN32
+void initialize_cvattr(conditionvariable_attr_t * cv_attr) {
+assert(cv_attr != NULL);
+pthread_condattr_init(_attr->cv_attr);
+cv_attr->initialized = 1;
+}
+#endif
+
+#if !defined(_WIN32) && !defined(__APPLE__)
+void condition_attr_set_clock(conditionvariable_attr_t * cv_attr, clockid_t
clock) {
+assert(cv_attr != NULL);
+pthread_condattr_setclock(_attr->cv_attr, clock);
+}
+#endif
+
+void initialize_cv(conditionvariable_t * cv, conditionvariable_attr_t *
cv_attr) {
+assert(cv != NULL);
+#ifndef WIN32
+if (cv_attr && cv_attr->initialized) {
+pthread_cond_init(>cv, _attr->cv_attr);
+}
+else {
+pthread_cond_init(>cv, NULL);
+}
+#else
+InitializeConditionVariable(>cv);
+#endif
+}
+
+void acquire_lock(lock_t * lock) {
+assert(lock != NULL);
+#ifndef WIN32
+pthread_mutex_lock(>mutex);
+#else
+EnterCriticalSection(>cs);
+#endif
+}
+
+void release_lock(lock_t * lock) {
+assert(lock != NULL);
+#ifndef WIN32
+pthread_mutex_unlock(>mutex);
+#else
+LeaveCriticalSection(>cs);
+#endif
+}
+
+#ifndef WIN32
+uint64_t get_time_millis(struct timespec ts) {
+ts.tv_sec += ts.tv_nsec / 10L;
+ts.tv_nsec = ts.tv_nsec % 10L;
+
+uint64_t ms = (ts.tv_sec * 1000) + (ts.tv_nsec / 100L);
+ts.tv_nsec = ts.tv_nsec % 100L;
+
+ms += lround((double)((double)ts.tv_nsec / 100L));
Review comment:
Missing `#include `
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371963185
##
File path: nanofi/src/coap/c2payload.c
##
@@ -0,0 +1,701 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+#include
+#include
+
+typedef enum {
+TYPE,
+NAME,
+INPUT,
+OUTPUT,
+IDENTIFIER,
+DISPLAYNAME,
+DESCRIPTION,
+VALIDATOR,
+SENSITIVE,
+DYNAMIC,
+REQUIRED,
+PROPERTIES,
+VERSION,
+STATUS,
+DEVICEINFO,
+SYSTEMFINO,
+MACHINEARCH,
+VCORES,
+PHYSICALMEMORYBYTES,
+NETWORKINFO,
+HOSTNAME,
+IPADDRESS,
+AGENTINFO,
+AGENTCLASS,
+UPTIME,
+AGENTMANIFEST,
+AGENTTYPE,
+IOMANIFEST,
+PROPERTYDESCRIPTORS,
+ECUINFO,
+OPERATION,
+OPERAND,
+FILEPATH,
+CHUNKSIZEBYTES,
+DELIMITER,
+TAILFREQMILLISECONDS,
+TCPPORT,
+NIFIPORT,
+REQUESTEDOPERATIONS
+} c2_keys_t;
+
+value_t value_uint8(uint8_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint8 = value;
+val.val_type = UINT8_TYPE;
+return val;
+}
+
+value_t value_uint16(uint16_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint16 = value;
+val.val_type = UINT16_TYPE;
+return val;
+}
+
+value_t value_uint32(uint32_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint32 = value;
+val.val_type = UINT32_TYPE;
+return val;
+}
+
+value_t value_uint64(uint64_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint64 = value;
+val.val_type = UINT64_TYPE;
+return val;
+}
+
+value_t value_string(const char * value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+copystr(value, _str);
+val.val_type = STRING_TYPE;
+return val;
+}
+
+value_t value_nstring(const unsigned char * value, size_t len) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_str = (char *)malloc(len + 1);
+memset(val.v_str, 0, len + 1);
+memcpy(val.v_str, value, len);
+val.val_type = STRING_TYPE;
+return val;
+}
+
+value_t value_map(c2_payload_map_t * value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_map = value;
+val.val_type = HASH_TYPE;
+return val;
+}
+
+value_t value_list(c2_payload_list_t * value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_maplist = value;
+val.val_type = LIST_TYPE;
+return val;
+}
+
+value_t value_property(properties_t * props) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_props = clone_properties(props);
+val.val_type = PROP_TYPE;
+return val;
+}
+
+value_t value_none() {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.val_type = NONE_TYPE;
+return val;
+}
+
+int is_value_none(value_t val) {
+return val.val_type == NONE_TYPE;
+}
+
+int is_value_uint(value_t val) {
+return val.val_type == UINT8_TYPE ||
+val.val_type == UINT16_TYPE ||
+val.val_type == UINT32_TYPE ||
+val.val_type == UINT64_TYPE;
+}
+
+int is_value_string(value_t val) {
+return val.val_type == STRING_TYPE;
+}
+
+int is_value_map(value_t val) {
+return val.val_type == HASH_TYPE;
+}
+
+int is_value_list(value_t val) {
+return val.val_type == LIST_TYPE;
+}
+
+int is_value_prop(value_t val) {
+return val.val_type == PROP_TYPE;
+}
+
+c2_payload_map_t * allocate_c2_payload() {
+c2_payload_map_t * map = (c2_payload_map_t
*)malloc(sizeof(c2_payload_map_t));
+memset(map, 0, sizeof(c2_payload_map_t));
+return map;
+}
+
+c2_payload_map_t * add_kvp(c2_payload_map_t ** map, uint16_t key, value_t
value) {
+c2_payload_map_t * field = allocate_c2_payload();
+field->key = key;
+field->value = value;
+HASH_ADD_INT((*map), key, field);
+return field;
+}
+
+c2_payload_list_t * add_list(c2_payload_list_t ** list, value_t value) {
+if (!list) return NULL;
+c2_payload_list_t * node = (c2_payload_list_t
*)malloc(sizeof(c2_payload_list_t));
+memset(node, 0, sizeof(c2_payload_list_t));
+node->value = value;
+LL_APPEND((*list), node);
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371916643
##
File path: nanofi/src/processors/file_input.c
##
@@ -0,0 +1,453 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+#ifndef WIN32
+#include
+#endif
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+void initialize_file_input(file_input_context_t * ctx) {
+initialize_lock(>stop_mutex);
+initialize_cv(>stop_cond, NULL);
+}
+
+void start_file_input(file_input_context_t * ctx) {
+acquire_lock(>stop_mutex);
+ctx->stop = 0;
+release_lock(>stop_mutex);
+}
+
+void stop_file_input(file_input_context_t * ctx) {
+ acquire_lock(>stop_mutex);
+ctx->stop = 1;
+condition_variable_broadcast(>stop_cond);
+release_lock(>stop_mutex);
+}
+
+#ifndef WIN32
+int validate_file_path(const char * file_path) {
+if (!file_path) {
+return -1;
+}
+
+ struct stat stats;
+ int ret = stat(file_path, );
+
+if (ret == -1) {
+logc(err, "Error occurred while getting file status {file: %s, error:
%s}\n", file_path, strerror(errno));
+return -1;
+}
+
+if (S_ISDIR(stats.st_mode)){
+logc(err, "Error: %s is a directory!\n", file_path);
+return -1;
+}
+return 0;
+}
+#else
+int validate_file_path(const char * file_path) {
+if (!file_path) {
+return -1;
+}
+HANDLE hFind;
+WIN32_FIND_DATA fd;
+
+hFind = FindFirstFile(file_path, );
+if (hFind == INVALID_HANDLE_VALUE || fd.dwFileAttributes ==
FILE_ATTRIBUTE_DIRECTORY) {
+return -1;
+}
+
+return 0;
+}
+#endif
+
+int validate_file_delimiter(const char * delimiter_str, char * delim) {
+if (!delimiter_str || strlen(delimiter_str) == 0) {
+return -1;
+}
+
+char delimiter[3];
+strncpy(delimiter, delimiter_str, 2);
+delimiter[2] = '\0';
+*delim = delimiter[0];
+
+if (*delim == '\\' && strlen(delimiter) > 1) {
+switch (delimiter[1]) {
+case 'r':
+*delim = '\r';
+break;
+case 'n':
+*delim = '\n';
+break;
+case 't':
+*delim = '\t';
+break;
+case '\\':
+*delim = '\\';
+break;
+}
+}
+return 0;
+}
+
+int validate_file_properties(struct file_input_context * context) {
+if (!context || !context->input_properties) {
+return -1;
+}
+
+properties_t * props = context->input_properties;
+properties_t * el = NULL;
+HASH_FIND_STR(props, "file_path", el);
+if (!el) {
+return -1;
+}
+char * file_path = el->value;
+if (!file_path) {
+return -1;
+}
+
+properties_t * cs = NULL;
+properties_t * dl = NULL;
+HASH_FIND_STR(props, "chunk_size_bytes", cs);
+HASH_FIND_STR(props, "delimiter", dl);
+if (dl && cs) {
+return -1;
+}
+
+if (!dl && !cs) {
+return -1;
+}
+
+char * chunk_size_str = NULL;
+char * delimiter = NULL;
+
+if (cs) {
+chunk_size_str = cs->value;
+}
+if (dl) {
+delimiter = dl->value;
+}
+
+el = NULL;
+HASH_FIND_STR(props, "tail_frequency_ms", el);
+if (!el) {
+return -1;
+}
+
+char * tail_frequency_str = el->value;
+
+uint64_t chunk_size_uint = 0;
+uint64_t tail_frequency_uint = 0;
+char delim = '\0';
+
+if ((validate_file_path(file_path) < 0)
+|| (dl && validate_file_delimiter(delimiter, ) < 0)
+|| (cs && str_to_uint(chunk_size_str, _size_uint) < 0)
+|| (str_to_uint(tail_frequency_str, _frequency_uint) < 0)
+|| (dl && delim == '\0')) {
+return -1;
+}
+
+//populate file input context with parameters
+size_t file_path_len = strlen(file_path);
+char * fp = context->file_path;
+if (fp) free(fp);
+context->file_path = (char *)malloc(file_path_len + 1);
+strcpy(context->file_path, file_path);
+
+context->tail_frequency_ms = tail_frequency_uint;
+
+if (cs)
+ context->chunk_size =
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration. URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371420564 ## File path: nanofi/include/core/string_utils.h ## @@ -87,6 +87,65 @@ int validate_list(token_list * tk_list); */ void attach_lists(token_list * to, token_list * from); +/** + * Copies source string into destination + * @param source, the string to copy from + * @param dest, the destination to copy into + * This function allocates heap memory for destination + * so the caller is expected to free it + * + * @attention the source string is expected to be + * null terminated + */ +void copystr(const char * source, char ** dest); Review comment: We could use the return value to return the result like `strdup` does. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371947617
##
File path: nanofi/src/coap/c2payload.c
##
@@ -0,0 +1,701 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+#include
+#include
+
+typedef enum {
+TYPE,
+NAME,
+INPUT,
+OUTPUT,
+IDENTIFIER,
+DISPLAYNAME,
+DESCRIPTION,
+VALIDATOR,
+SENSITIVE,
+DYNAMIC,
+REQUIRED,
+PROPERTIES,
+VERSION,
+STATUS,
+DEVICEINFO,
+SYSTEMFINO,
+MACHINEARCH,
+VCORES,
+PHYSICALMEMORYBYTES,
+NETWORKINFO,
+HOSTNAME,
+IPADDRESS,
+AGENTINFO,
+AGENTCLASS,
+UPTIME,
+AGENTMANIFEST,
+AGENTTYPE,
+IOMANIFEST,
+PROPERTYDESCRIPTORS,
+ECUINFO,
+OPERATION,
+OPERAND,
+FILEPATH,
+CHUNKSIZEBYTES,
+DELIMITER,
+TAILFREQMILLISECONDS,
+TCPPORT,
+NIFIPORT,
+REQUESTEDOPERATIONS
+} c2_keys_t;
+
+value_t value_uint8(uint8_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint8 = value;
+val.val_type = UINT8_TYPE;
+return val;
+}
+
+value_t value_uint16(uint16_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint16 = value;
+val.val_type = UINT16_TYPE;
+return val;
+}
+
+value_t value_uint32(uint32_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint32 = value;
+val.val_type = UINT32_TYPE;
+return val;
+}
+
+value_t value_uint64(uint64_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint64 = value;
+val.val_type = UINT64_TYPE;
+return val;
+}
+
+value_t value_string(const char * value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+copystr(value, _str);
+val.val_type = STRING_TYPE;
+return val;
+}
+
+value_t value_nstring(const unsigned char * value, size_t len) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_str = (char *)malloc(len + 1);
+memset(val.v_str, 0, len + 1);
+memcpy(val.v_str, value, len);
+val.val_type = STRING_TYPE;
+return val;
+}
+
+value_t value_map(c2_payload_map_t * value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_map = value;
+val.val_type = HASH_TYPE;
+return val;
+}
+
+value_t value_list(c2_payload_list_t * value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_maplist = value;
+val.val_type = LIST_TYPE;
+return val;
+}
+
+value_t value_property(properties_t * props) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_props = clone_properties(props);
+val.val_type = PROP_TYPE;
+return val;
+}
+
+value_t value_none() {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.val_type = NONE_TYPE;
+return val;
+}
+
+int is_value_none(value_t val) {
+return val.val_type == NONE_TYPE;
+}
+
+int is_value_uint(value_t val) {
+return val.val_type == UINT8_TYPE ||
+val.val_type == UINT16_TYPE ||
+val.val_type == UINT32_TYPE ||
+val.val_type == UINT64_TYPE;
+}
+
+int is_value_string(value_t val) {
+return val.val_type == STRING_TYPE;
+}
+
+int is_value_map(value_t val) {
+return val.val_type == HASH_TYPE;
+}
+
+int is_value_list(value_t val) {
+return val.val_type == LIST_TYPE;
+}
+
+int is_value_prop(value_t val) {
+return val.val_type == PROP_TYPE;
+}
+
+c2_payload_map_t * allocate_c2_payload() {
+c2_payload_map_t * map = (c2_payload_map_t
*)malloc(sizeof(c2_payload_map_t));
+memset(map, 0, sizeof(c2_payload_map_t));
+return map;
+}
+
+c2_payload_map_t * add_kvp(c2_payload_map_t ** map, uint16_t key, value_t
value) {
+c2_payload_map_t * field = allocate_c2_payload();
+field->key = key;
+field->value = value;
+HASH_ADD_INT((*map), key, field);
+return field;
+}
+
+c2_payload_list_t * add_list(c2_payload_list_t ** list, value_t value) {
+if (!list) return NULL;
+c2_payload_list_t * node = (c2_payload_list_t
*)malloc(sizeof(c2_payload_list_t));
+memset(node, 0, sizeof(c2_payload_list_t));
+node->value = value;
+LL_APPEND((*list), node);
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371415224
##
File path: nanofi/ecu/tailfile_ecu_c2.c
##
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+#ifndef _WIN32
+#include
+#endif
+#include
+#include
+#include
+#include
+#include
+#include
+
+volatile sig_atomic_t stop_ecu = 0;
+
+void ecu_signal_handler(int signum) {
+if (signum == SIGINT || signum == SIGTERM) {
+stop_ecu = 1;
+}
+}
+
+void setup_ecu_signal_action() {
+#ifdef _WIN32
Review comment:
Why do we mostly use `WIN32` but sometimes `_WIN32`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371920476
##
File path: nanofi/src/processors/file_input.c
##
@@ -0,0 +1,453 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+#ifndef WIN32
+#include
+#endif
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+void initialize_file_input(file_input_context_t * ctx) {
+initialize_lock(>stop_mutex);
+initialize_cv(>stop_cond, NULL);
+}
+
+void start_file_input(file_input_context_t * ctx) {
+acquire_lock(>stop_mutex);
+ctx->stop = 0;
+release_lock(>stop_mutex);
+}
+
+void stop_file_input(file_input_context_t * ctx) {
+ acquire_lock(>stop_mutex);
+ctx->stop = 1;
+condition_variable_broadcast(>stop_cond);
+release_lock(>stop_mutex);
+}
+
+#ifndef WIN32
+int validate_file_path(const char * file_path) {
+if (!file_path) {
+return -1;
+}
+
+ struct stat stats;
+ int ret = stat(file_path, );
+
+if (ret == -1) {
+logc(err, "Error occurred while getting file status {file: %s, error:
%s}\n", file_path, strerror(errno));
+return -1;
+}
+
+if (S_ISDIR(stats.st_mode)){
+logc(err, "Error: %s is a directory!\n", file_path);
+return -1;
+}
+return 0;
+}
+#else
+int validate_file_path(const char * file_path) {
+if (!file_path) {
+return -1;
+}
+HANDLE hFind;
+WIN32_FIND_DATA fd;
+
+hFind = FindFirstFile(file_path, );
+if (hFind == INVALID_HANDLE_VALUE || fd.dwFileAttributes ==
FILE_ATTRIBUTE_DIRECTORY) {
+return -1;
+}
+
+return 0;
+}
+#endif
+
+int validate_file_delimiter(const char * delimiter_str, char * delim) {
+if (!delimiter_str || strlen(delimiter_str) == 0) {
+return -1;
+}
+
+char delimiter[3];
+strncpy(delimiter, delimiter_str, 2);
+delimiter[2] = '\0';
+*delim = delimiter[0];
+
+if (*delim == '\\' && strlen(delimiter) > 1) {
+switch (delimiter[1]) {
+case 'r':
+*delim = '\r';
+break;
+case 'n':
+*delim = '\n';
+break;
+case 't':
+*delim = '\t';
+break;
+case '\\':
+*delim = '\\';
+break;
+}
+}
+return 0;
+}
+
+int validate_file_properties(struct file_input_context * context) {
+if (!context || !context->input_properties) {
+return -1;
+}
+
+properties_t * props = context->input_properties;
+properties_t * el = NULL;
+HASH_FIND_STR(props, "file_path", el);
+if (!el) {
+return -1;
+}
+char * file_path = el->value;
+if (!file_path) {
+return -1;
+}
+
+properties_t * cs = NULL;
+properties_t * dl = NULL;
+HASH_FIND_STR(props, "chunk_size_bytes", cs);
+HASH_FIND_STR(props, "delimiter", dl);
+if (dl && cs) {
+return -1;
+}
+
+if (!dl && !cs) {
+return -1;
+}
+
+char * chunk_size_str = NULL;
+char * delimiter = NULL;
+
+if (cs) {
+chunk_size_str = cs->value;
+}
+if (dl) {
+delimiter = dl->value;
+}
+
+el = NULL;
+HASH_FIND_STR(props, "tail_frequency_ms", el);
+if (!el) {
+return -1;
+}
+
+char * tail_frequency_str = el->value;
+
+uint64_t chunk_size_uint = 0;
+uint64_t tail_frequency_uint = 0;
+char delim = '\0';
+
+if ((validate_file_path(file_path) < 0)
+|| (dl && validate_file_delimiter(delimiter, ) < 0)
+|| (cs && str_to_uint(chunk_size_str, _size_uint) < 0)
+|| (str_to_uint(tail_frequency_str, _frequency_uint) < 0)
+|| (dl && delim == '\0')) {
+return -1;
+}
+
+//populate file input context with parameters
+size_t file_path_len = strlen(file_path);
+char * fp = context->file_path;
+if (fp) free(fp);
+context->file_path = (char *)malloc(file_path_len + 1);
+strcpy(context->file_path, file_path);
+
+context->tail_frequency_ms = tail_frequency_uint;
+
+if (cs)
+ context->chunk_size =
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371388203
##
File path: nanofi/src/c2_api/c2api.c
##
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+#include "uthash.h"
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+int is_little_endian() {
+const uint16_t x = 1;
+uint8_t * y = (uint8_t *)
+return *y == 1;
+}
Review comment:
* CPU byte ordering can be determined at compile-time. This is how abseil
(core utilities by google) does it:
https://github.com/abseil/abseil-cpp/blob/master/absl/base/config.h#L433
* The above way of type punning violates the aliasing rules of C and invokes
undefined behavior. Use union-based type punning (C only, might not work on
MSVC) or `memcpy` to `char[]`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371960273
##
File path: nanofi/src/core/threadutils.c
##
@@ -0,0 +1,56 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include
+#include
+
+int create_thread(thread_handle_t * hnd, thread_proc_t tproc, void * args) {
+ if (!hnd || tproc.threadfunc == NULL) {
+ return -1;
+ }
+
+#ifndef WIN32
+ if (pthread_create(>thread, NULL, tproc.threadfunc, args) != 0) {
+ return -1;
+ }
+#else
+ uintptr_t ret = _beginthreadex(NULL, 0, tproc.threadfunc, args, 0,
NULL);
+ if (ret == 0) {
+ hnd->thread = 0;
+ return -1;
+ }
+ hnd->thread = ret;
+#endif
+ return 0;
+}
+
+void wait_thread_complete(thread_handle_t * hnd) {
+ assert(hnd != NULL);
+#ifndef WIN32
+ pthread_join(hnd->thread, NULL);
+#else
+ WaitForSingleObject((void *)(>thread), INFINITE);
+#endif
+}
+
+void thread_sleep_ms(uint64_t millis) {
+#ifndef WIN32
+ usleep(millis * 1000L);
Review comment:
Missing `#include `
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371821508
##
File path: nanofi/src/core/file_utils.c
##
@@ -139,3 +200,46 @@ char * get_current_working_directory() {
free(cwd);
return NULL;
}
+
+properties_t * read_configuration_file(const char * file_path) {
+if (!file_path) {
+logc(err, "%s", "No file path provided");
+return NULL;
+}
+
+properties_t * params = NULL;
+FILE * fp = fopen(file_path, "r");
+char * line = NULL;
+size_t size = 0;
+if (!fp) {
+logc(err, "Could not open file %s", file_path);
+return NULL;
+}
+#ifndef WIN32
+while (getline(, , fp) > 0) {
+#else
+size = 1024;
+line = (char *)malloc(1024);
+while (fgets(line, size, fp) != NULL) {
+#endif
+char ** tokens = parse_tokens(line, size, 2, " =\n");
+properties_t * el = (properties_t *)malloc(sizeof(properties_t));
+
+el->key = tokens[0];
+el->value = tokens[1];
+
+char ** tmp = tokens;
+free(tmp);
Review comment:
What's the purpose of this temporary?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371860515
##
File path: nanofi/src/ecu_api/ecuapi.c
##
@@ -0,0 +1,674 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+#include
+#include
+#include
+#include
+#include
+
+int validate_io_type(io_type_t ip, io_type_t op) {
+return ip >= TAILFILE && ip <= MANUAL && op >= TAILFILE && op < MQTTIO;
+}
+
+int initialize_ecu(ecu_context_t * ecu, const char * name, input_context_t *
ip, output_context_t * op) {
+if (!ecu) return -1;
+
+if (!validate_io_type(ip->type, op->type)) {
+logc(err, "Input/Output is out of range. Valid range is %d to %d",
TAILFILE, MQTTIO);
+return -1;
+}
+
+if (name && strlen(name) > 0) {
+size_t len = strlen(name);
+ecu->name = (char *)malloc(len + 1);
+strcpy(ecu->name, name);
+}
+
+CIDGenerator gen;
+gen.implementation_ = CUUID_DEFAULT_IMPL;
+generate_uuid(, ecu->uuid);
+ecu->uuid[36] = '\0';
+
+initialize_lock(>ctx_lock);
+ecu->input = ip;
+ecu->output = op;
+return 0;
+}
+
+ecu_context_t * allocate_ecu() {
+ecu_context_t * ecu_ctx = (ecu_context_t *)malloc(sizeof(struct
ecu_context));
+memset(ecu_ctx, 0, sizeof(struct ecu_context));
+return ecu_ctx;
+}
+
+void free_input(input_context_t * input) {
+if (input_map[input->type].free_input_context) {
+void * ip_ctx = input->proc_ctx;
+input->proc_ctx = NULL;
+input_map[input->type].free_input_context(ip_ctx);
+}
+}
+
+void free_output(output_context_t * output) {
+if (output_map[output->type].free_output_context) {
+void * op_ctx = output->proc_ctx;
+output->proc_ctx = NULL;
+output_map[output->type].free_output_context(op_ctx);
+}
+}
+
+void free_ecu_context(ecu_context_t * ctx) {
+if (!ctx) return;
+free_input(ctx->input);
+free_output(ctx->output);
+free(ctx->input);
+free(ctx->output);
+}
+
+void free_property(properties_t * prop) {
+if (prop) {
+free(prop->key);
+free(prop->value);
+}
+}
+
+void free_properties(properties_t * prop) {
+if (prop) {
+properties_t * el, *tmp = NULL;
+HASH_ITER(hh, prop, el, tmp) {
+HASH_DEL(prop, el);
+free(el->key);
+free(el->value);
+free(el);
+}
+}
+}
+
+int add_property(struct properties ** head, const char * name, const char *
value) {
+if (!head || !name || !value) {
+return -1;
+}
+properties_t * el = NULL;
+HASH_FIND_STR(*head, name, el);
+if (el) {
+HASH_DEL(*head, el);
+free_property(el);
+free(el);
+}
+
+properties_t * new_prop = (properties_t *) malloc(sizeof(struct
properties));
+size_t name_len = strlen(name);
+size_t value_len = strlen(value);
+new_prop->key = (char *) malloc(name_len + 1);
+memset(new_prop->key, 0, name_len + 1);
+strcpy(new_prop->key, name);
+
+new_prop->value = (char *) malloc(value_len + 1);
+memset(new_prop->value, 0, value_len + 1);
+strcpy(new_prop->value, value);
+
+HASH_ADD_KEYPTR(hh, *head, new_prop->key, strlen(new_prop->key), new_prop);
+return 0;
+}
+
+int set_ecu_input_property(ecu_context_t * ecu, const char * name, const char
* value) {
+if (!ecu || !name || !value) {
+return -1;
+}
+return set_input_property(ecu->input, name, value);
+}
+
+int set_ecu_output_property(ecu_context_t * ecu, const char * name, const char
* value) {
+if (!ecu || !name || !value) {
+return -1;
+}
+return set_output_property(ecu->output, name, value);
+}
+
+int set_input_properties(ecu_context_t * ecu_ctx, properties_t * props) {
+if (!ecu_ctx || !props) {
+return -1;
+}
+properties_t * el, *tmp = NULL;
+HASH_ITER(hh, props, el, tmp) {
+if (set_ecu_input_property(ecu_ctx, el->key, el->value) < 0) {
+return -1;
+}
+}
+return 0;
+}
+
+int set_output_properties(ecu_context_t * ecu_ctx, properties_t * props) {
+if (!ecu_ctx
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371943442
##
File path: nanofi/src/coap/c2payload.c
##
@@ -0,0 +1,701 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+#include
+#include
+
+typedef enum {
+TYPE,
+NAME,
+INPUT,
+OUTPUT,
+IDENTIFIER,
+DISPLAYNAME,
+DESCRIPTION,
+VALIDATOR,
+SENSITIVE,
+DYNAMIC,
+REQUIRED,
+PROPERTIES,
+VERSION,
+STATUS,
+DEVICEINFO,
+SYSTEMFINO,
+MACHINEARCH,
+VCORES,
+PHYSICALMEMORYBYTES,
+NETWORKINFO,
+HOSTNAME,
+IPADDRESS,
+AGENTINFO,
+AGENTCLASS,
+UPTIME,
+AGENTMANIFEST,
+AGENTTYPE,
+IOMANIFEST,
+PROPERTYDESCRIPTORS,
+ECUINFO,
+OPERATION,
+OPERAND,
+FILEPATH,
+CHUNKSIZEBYTES,
+DELIMITER,
+TAILFREQMILLISECONDS,
+TCPPORT,
+NIFIPORT,
+REQUESTEDOPERATIONS
+} c2_keys_t;
+
+value_t value_uint8(uint8_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint8 = value;
+val.val_type = UINT8_TYPE;
+return val;
+}
+
+value_t value_uint16(uint16_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint16 = value;
+val.val_type = UINT16_TYPE;
+return val;
+}
+
+value_t value_uint32(uint32_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint32 = value;
+val.val_type = UINT32_TYPE;
+return val;
+}
+
+value_t value_uint64(uint64_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint64 = value;
+val.val_type = UINT64_TYPE;
+return val;
+}
+
+value_t value_string(const char * value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+copystr(value, _str);
+val.val_type = STRING_TYPE;
+return val;
+}
+
+value_t value_nstring(const unsigned char * value, size_t len) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_str = (char *)malloc(len + 1);
+memset(val.v_str, 0, len + 1);
+memcpy(val.v_str, value, len);
+val.val_type = STRING_TYPE;
+return val;
+}
+
+value_t value_map(c2_payload_map_t * value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_map = value;
+val.val_type = HASH_TYPE;
+return val;
+}
+
+value_t value_list(c2_payload_list_t * value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_maplist = value;
+val.val_type = LIST_TYPE;
+return val;
+}
+
+value_t value_property(properties_t * props) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_props = clone_properties(props);
+val.val_type = PROP_TYPE;
+return val;
+}
+
+value_t value_none() {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.val_type = NONE_TYPE;
+return val;
+}
+
+int is_value_none(value_t val) {
+return val.val_type == NONE_TYPE;
+}
+
+int is_value_uint(value_t val) {
+return val.val_type == UINT8_TYPE ||
+val.val_type == UINT16_TYPE ||
+val.val_type == UINT32_TYPE ||
+val.val_type == UINT64_TYPE;
+}
+
+int is_value_string(value_t val) {
+return val.val_type == STRING_TYPE;
+}
+
+int is_value_map(value_t val) {
+return val.val_type == HASH_TYPE;
+}
+
+int is_value_list(value_t val) {
+return val.val_type == LIST_TYPE;
+}
+
+int is_value_prop(value_t val) {
+return val.val_type == PROP_TYPE;
+}
+
+c2_payload_map_t * allocate_c2_payload() {
+c2_payload_map_t * map = (c2_payload_map_t
*)malloc(sizeof(c2_payload_map_t));
+memset(map, 0, sizeof(c2_payload_map_t));
+return map;
+}
+
+c2_payload_map_t * add_kvp(c2_payload_map_t ** map, uint16_t key, value_t
value) {
+c2_payload_map_t * field = allocate_c2_payload();
+field->key = key;
+field->value = value;
+HASH_ADD_INT((*map), key, field);
+return field;
+}
+
+c2_payload_list_t * add_list(c2_payload_list_t ** list, value_t value) {
+if (!list) return NULL;
+c2_payload_list_t * node = (c2_payload_list_t
*)malloc(sizeof(c2_payload_list_t));
+memset(node, 0, sizeof(c2_payload_list_t));
+node->value = value;
+LL_APPEND((*list), node);
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371959937
##
File path: nanofi/src/core/synchutils.c
##
@@ -0,0 +1,189 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+#include
+#include
+#include
+
+void initialize_lock(lock_t * lock) {
+#ifndef WIN32
+pthread_mutex_init(>mutex, NULL);
+#else
+InitializeCriticalSection(>cs);
+#endif
+}
+
+#ifndef WIN32
+void initialize_cvattr(conditionvariable_attr_t * cv_attr) {
+assert(cv_attr != NULL);
+pthread_condattr_init(_attr->cv_attr);
+cv_attr->initialized = 1;
+}
+#endif
+
+#if !defined(_WIN32) && !defined(__APPLE__)
+void condition_attr_set_clock(conditionvariable_attr_t * cv_attr, clockid_t
clock) {
+assert(cv_attr != NULL);
+pthread_condattr_setclock(_attr->cv_attr, clock);
+}
+#endif
+
+void initialize_cv(conditionvariable_t * cv, conditionvariable_attr_t *
cv_attr) {
+assert(cv != NULL);
+#ifndef WIN32
+if (cv_attr && cv_attr->initialized) {
+pthread_cond_init(>cv, _attr->cv_attr);
+}
+else {
+pthread_cond_init(>cv, NULL);
+}
+#else
+InitializeConditionVariable(>cv);
+#endif
+}
+
+void acquire_lock(lock_t * lock) {
+assert(lock != NULL);
+#ifndef WIN32
+pthread_mutex_lock(>mutex);
+#else
+EnterCriticalSection(>cs);
+#endif
+}
+
+void release_lock(lock_t * lock) {
+assert(lock != NULL);
+#ifndef WIN32
+pthread_mutex_unlock(>mutex);
+#else
+LeaveCriticalSection(>cs);
+#endif
+}
+
+#ifndef WIN32
+uint64_t get_time_millis(struct timespec ts) {
+ts.tv_sec += ts.tv_nsec / 10L;
+ts.tv_nsec = ts.tv_nsec % 10L;
+
+uint64_t ms = (ts.tv_sec * 1000) + (ts.tv_nsec / 100L);
+ts.tv_nsec = ts.tv_nsec % 100L;
+
+ms += lround((double)((double)ts.tv_nsec / 100L));
+return ms;
+}
+
+struct timespec get_timespec_millis_from_now(uint64_t millis) {
+struct timespec ts;
+memset(, 0, sizeof(ts));
Review comment:
Missing `#include `
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371750620
##
File path: extensions/coap/nanofi/coap_functions.c
##
@@ -146,29 +209,80 @@ void no_acknowledgement(struct coap_context_t *ctx,
coap_session_t *session, coa
}
}
-void response_handler(struct coap_context_t *ctx, struct coap_session_t
*session, coap_pdu_t *sent, coap_pdu_t *received, const coap_tid_t id) {
- unsigned char* data;
- size_t data_len;
- coap_opt_iterator_t opt_iter;
- coap_opt_t * block_opt = coap_check_option(received, COAP_OPTION_BLOCK1,
_iter);
- if (block_opt) {
-printf("Block option not currently supported");
- } else {
-if (!global_ptrs.data_received) {
- return;
+void get_pdu_optlist(coap_pdu_t * pdu, coap_optlist_t ** optlist) {
+coap_opt_t *option;
+coap_opt_iterator_t opt_iter;
+coap_option_iterator_init(pdu, _iter, COAP_OPT_ALL);
+while ((option = coap_option_next(_iter))) {
+//while iterating create a coap_optlist_t
+if (opt_iter.type != COAP_OPTION_BLOCK1 && opt_iter.type !=
COAP_OPTION_SIZE1)
+coap_insert_optlist(optlist, coap_new_optlist(opt_iter.type,
coap_opt_length(option), coap_opt_value(option)));
}
+}
+
+void handle_block_response(coap_opt_t * block_opt, struct coap_context_t *ctx,
struct coap_session_t *session, coap_pdu_t *sent) {
+unsigned int szx = COAP_OPT_BLOCK_SZX(block_opt);
+unsigned int num = coap_opt_block_num(block_opt);
+coap_block_t block;
+block.num = num;
+block.szx = szx;
+//We are handling a response to POST request. Server never uses BLOCK1
option in the response
+//unless client initiated BLOCK1 blockwise transfer request. Therefore, it
is safe to consider
+//that the recently sent pdu within this session has the BLOCK1 option
+coap_opt_iterator_t opt_iter;
+block_opt = coap_check_option(sent, COAP_OPTION_BLOCK1, _iter);
+if (szx != COAP_OPT_BLOCK_SZX(block_opt)) {
+block.num = coap_opt_block_num(block_opt);
+block.szx = COAP_OPT_BLOCK_SZX(block_opt);
+//Server negotiated a different block size
+//client will follow the server, but we have to adjust block number
+unsigned int bytes_sent = ((block.num + 1) << (block.szx + 4));
+if (bytes_sent % (1 << (szx + 4)) == 0) {
+num = block.num = (bytes_sent >> (szx + 4)) - 1;
Review comment:
`szx + 4` looks like a magic number that deserves a name. Could you give it
a name?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371961573
##
File path: nanofi/include/coap/coapprotocol.h
##
@@ -0,0 +1,49 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef NIFI_MINIFI_CPP_COAPPROTOCOL_H
+#define NIFI_MINIFI_CPP_COAPPROTOCOL_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include
+#include "nanofi/coap_message.h"
Review comment:
I've got a "No such file or directory" for this header. Please adapt the
include path or change to `#include "coap_message.h"`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371962298
##
File path: nanofi/src/coap/c2payload.c
##
@@ -0,0 +1,701 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+#include
+#include
+
+typedef enum {
+TYPE,
+NAME,
+INPUT,
+OUTPUT,
+IDENTIFIER,
+DISPLAYNAME,
+DESCRIPTION,
+VALIDATOR,
+SENSITIVE,
+DYNAMIC,
+REQUIRED,
+PROPERTIES,
+VERSION,
+STATUS,
+DEVICEINFO,
+SYSTEMFINO,
+MACHINEARCH,
+VCORES,
+PHYSICALMEMORYBYTES,
+NETWORKINFO,
+HOSTNAME,
+IPADDRESS,
+AGENTINFO,
+AGENTCLASS,
+UPTIME,
+AGENTMANIFEST,
+AGENTTYPE,
+IOMANIFEST,
+PROPERTYDESCRIPTORS,
+ECUINFO,
+OPERATION,
+OPERAND,
+FILEPATH,
+CHUNKSIZEBYTES,
+DELIMITER,
+TAILFREQMILLISECONDS,
+TCPPORT,
+NIFIPORT,
+REQUESTEDOPERATIONS
+} c2_keys_t;
+
+value_t value_uint8(uint8_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint8 = value;
+val.val_type = UINT8_TYPE;
+return val;
+}
+
+value_t value_uint16(uint16_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint16 = value;
+val.val_type = UINT16_TYPE;
+return val;
+}
+
+value_t value_uint32(uint32_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint32 = value;
+val.val_type = UINT32_TYPE;
+return val;
+}
+
+value_t value_uint64(uint64_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint64 = value;
+val.val_type = UINT64_TYPE;
+return val;
+}
+
+value_t value_string(const char * value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+copystr(value, _str);
Review comment:
`copystr` is undeclared. Add `#include "core/string_utils.h"`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371827789
##
File path: nanofi/src/core/ring_buffer.c
##
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+#include
+
+size_t min(size_t x, size_t y) {
Review comment:
Functions private to the compilation unit should have `static` linkage
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371957567
##
File path: nanofi/src/core/message_queue.c
##
@@ -0,0 +1,276 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+#include
+#include
+#include
+#include
+
+message_queue_t * create_msg_queue(uint64_t capacity_bytes) {
+ring_buffer_t * rb = (ring_buffer_t *)malloc(sizeof(ring_buffer_t));
+memset(rb, 0, sizeof(ring_buffer_t));
+rb->capacity = capacity_bytes;
+rb->data = (char *)malloc(rb->capacity);
+message_queue_t * mq = (message_queue_t *)malloc(sizeof(message_queue_t));
+memset(mq, 0, sizeof(message_queue_t));
+mq->ring_buff = rb;
+initialize_lock(>queue_lock);
+#ifndef WIN32
+initialize_cvattr(>wrt_notify_attr);
+#ifndef __APPLE__
+condition_attr_set_clock(>wrt_notify_attr, CLOCK_MONOTONIC);
+#endif
+initialize_cv(>write_notify, >wrt_notify_attr);
+#else
+initialize_cv(>write_notify, NULL);
+#endif
+return mq;
+}
+
+void set_attribute_update_cb(message_queue_t * mq, attribute_set_cb_t cb) {
+mq->attr_cb = cb;
+}
+
+void free_queue(message_queue_t * mq) {
+if (!mq) return;
+acquire_lock(>queue_lock);
+message_attrs_t * head = mq->attrs;
+while (head) {
+message_attrs_t * tmp = head;
+head = head->next;
+free_attributes(tmp->as);
+free(tmp);
+}
+free_ring_buffer(mq->ring_buff);
+destroy_lock(>queue_lock);
+destroy_cvattr(>wrt_notify_attr);
+destroy_cv(>write_notify);
+free(mq);
+}
+
+attribute_set prepare_attributes(properties_t * attributes) {
+attribute_set as;
+memset(, 0, sizeof(attribute_set));
+if (!attributes) return as;
+
+as.size = HASH_COUNT(attributes);
+attribute * attrs = (attribute *)malloc(as.size * sizeof(attribute));
+
+properties_t *p, *tmp;
+int i = 0;
+HASH_ITER(hh, attributes, p, tmp) {
+attrs[i].key = (char *)malloc(strlen(p->key) + 1);
+strcpy(attrs[i].key, p->key);
Review comment:
The `const` qualifier on `attrs[i].key` is ignored. We should first create
the string as `char*` (non-`const`) then assign it to `attrs[i].key`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371416062
##
File path: CMakeLists.txt
##
@@ -312,8 +312,8 @@ if (ENABLE_ALL OR ENABLE_COAP STREQUAL "ON")
use_bundled_libcoap(${CMAKE_CURRENT_SOURCE_DIR}
${CMAKE_CURRENT_BINARY_DIR})
createExtension(COAP-EXTENSION "COAP EXTENSIONS" "Enables LibCOAP
Functionality." "extensions/coap" "extensions/coap/tests/")
if( NOT DISABLE_CURL)
- add_dependencies(minifi-coap minifi-http-curl)
- endif()
+ add_dependencies(minifi-coap minifi-http-curl)
+endif()
Review comment:
The file is indented with tabs. Why was this changed on these lines but not
on the rest of the file?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371944211
##
File path: nanofi/src/coap/c2payload.c
##
@@ -0,0 +1,701 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+#include
+#include
+
+typedef enum {
+TYPE,
+NAME,
+INPUT,
+OUTPUT,
+IDENTIFIER,
+DISPLAYNAME,
+DESCRIPTION,
+VALIDATOR,
+SENSITIVE,
+DYNAMIC,
+REQUIRED,
+PROPERTIES,
+VERSION,
+STATUS,
+DEVICEINFO,
+SYSTEMFINO,
+MACHINEARCH,
+VCORES,
+PHYSICALMEMORYBYTES,
+NETWORKINFO,
+HOSTNAME,
+IPADDRESS,
+AGENTINFO,
+AGENTCLASS,
+UPTIME,
+AGENTMANIFEST,
+AGENTTYPE,
+IOMANIFEST,
+PROPERTYDESCRIPTORS,
+ECUINFO,
+OPERATION,
+OPERAND,
+FILEPATH,
+CHUNKSIZEBYTES,
+DELIMITER,
+TAILFREQMILLISECONDS,
+TCPPORT,
+NIFIPORT,
+REQUESTEDOPERATIONS
+} c2_keys_t;
+
+value_t value_uint8(uint8_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint8 = value;
+val.val_type = UINT8_TYPE;
+return val;
+}
+
+value_t value_uint16(uint16_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint16 = value;
+val.val_type = UINT16_TYPE;
+return val;
+}
+
+value_t value_uint32(uint32_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint32 = value;
+val.val_type = UINT32_TYPE;
+return val;
+}
+
+value_t value_uint64(uint64_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint64 = value;
+val.val_type = UINT64_TYPE;
+return val;
+}
+
+value_t value_string(const char * value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+copystr(value, _str);
+val.val_type = STRING_TYPE;
+return val;
+}
+
+value_t value_nstring(const unsigned char * value, size_t len) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_str = (char *)malloc(len + 1);
+memset(val.v_str, 0, len + 1);
+memcpy(val.v_str, value, len);
+val.val_type = STRING_TYPE;
+return val;
+}
+
+value_t value_map(c2_payload_map_t * value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_map = value;
+val.val_type = HASH_TYPE;
+return val;
+}
+
+value_t value_list(c2_payload_list_t * value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_maplist = value;
+val.val_type = LIST_TYPE;
+return val;
+}
+
+value_t value_property(properties_t * props) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_props = clone_properties(props);
+val.val_type = PROP_TYPE;
+return val;
+}
+
+value_t value_none() {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.val_type = NONE_TYPE;
+return val;
+}
+
+int is_value_none(value_t val) {
+return val.val_type == NONE_TYPE;
+}
+
+int is_value_uint(value_t val) {
+return val.val_type == UINT8_TYPE ||
+val.val_type == UINT16_TYPE ||
+val.val_type == UINT32_TYPE ||
+val.val_type == UINT64_TYPE;
+}
+
+int is_value_string(value_t val) {
+return val.val_type == STRING_TYPE;
+}
+
+int is_value_map(value_t val) {
+return val.val_type == HASH_TYPE;
+}
+
+int is_value_list(value_t val) {
+return val.val_type == LIST_TYPE;
+}
+
+int is_value_prop(value_t val) {
+return val.val_type == PROP_TYPE;
+}
+
+c2_payload_map_t * allocate_c2_payload() {
+c2_payload_map_t * map = (c2_payload_map_t
*)malloc(sizeof(c2_payload_map_t));
+memset(map, 0, sizeof(c2_payload_map_t));
+return map;
+}
+
+c2_payload_map_t * add_kvp(c2_payload_map_t ** map, uint16_t key, value_t
value) {
+c2_payload_map_t * field = allocate_c2_payload();
+field->key = key;
+field->value = value;
+HASH_ADD_INT((*map), key, field);
+return field;
+}
+
+c2_payload_list_t * add_list(c2_payload_list_t ** list, value_t value) {
+if (!list) return NULL;
+c2_payload_list_t * node = (c2_payload_list_t
*)malloc(sizeof(c2_payload_list_t));
+memset(node, 0, sizeof(c2_payload_list_t));
+node->value = value;
+LL_APPEND((*list), node);
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371763092
##
File path: nanofi/ecu/c2_server.c
##
@@ -0,0 +1,293 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+/**
+ * The flow of C2 operations looks as follows
+ * |***||**| c2 operations
||
+ * | | c2 operations | |--->|
|
+ * | c2 client |--->| c2 server|<---| c2 agent /
ecu |
+ * | || | c2 heartbeat |
|
+ * *
**
+ *
+ *
+ * This is meant to be used as a c2 server to listen
+ * for heartbeats from c2 agents.
+ * It keeps track of c2 agents by the agent's uuid
+ * and when c2 command is received from a c2 client,
+ * forwards that command to the corresponding c2 agent
+ * in response to the heartbeat
+ *
+ * This is in no way meant to be used in production.
+ * This is just a skeleton/dummy c2 server used to
+ * test c2 agents functionality.
+ */
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include "api/ecu.h"
+#include "utlist.h"
+
+#include
+
+
+volatile sig_atomic_t stop_c2 = 0;
+
+void c2_signal_handler(int signum) {
+if (signum == SIGINT || signum == SIGTERM) {
+stop_c2 = 1;
+}
+}
+
+void setup_c2_signal_action() {
+#ifdef _WIN32
+signal(SIGINT, c2_signal_handler);
+signal(SIGTERM, c2_signal_handler);
+#else
+struct sigaction action;
+memset(, 0, sizeof(sigaction));
+action.sa_handler = c2_signal_handler;
+sigaction(SIGTERM, , NULL);
+sigaction(SIGINT, , NULL);
+#endif
+}
+
+typedef struct c2_server_responses {
+char uuid[37]; //key
+c2_server_response_t * response;
+UT_hash_handle hh;
+} c2_server_responses_t;
+
+typedef struct agents {
+char uuid[37];
+struct agents * next;
+} agents_t;
+
+int little_endian = 0;
+c2_server_responses_t * responses = NULL;
+agents_t * ags = NULL;
+
+int find_agent(char * uuid_str) {
+agents_t * el;
+LL_FOREACH(ags, el) {
+if (memcmp(el->uuid, uuid_str, strlen(uuid_str)) == 0) {
Review comment:
`memcmp` + `strlen` = `strcmp`
Why are we not using `strcmp` for string comparison?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371945035
##
File path: nanofi/src/coap/c2payload.c
##
@@ -0,0 +1,701 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+#include
+#include
+
+typedef enum {
+TYPE,
+NAME,
+INPUT,
+OUTPUT,
+IDENTIFIER,
+DISPLAYNAME,
+DESCRIPTION,
+VALIDATOR,
+SENSITIVE,
+DYNAMIC,
+REQUIRED,
+PROPERTIES,
+VERSION,
+STATUS,
+DEVICEINFO,
+SYSTEMFINO,
+MACHINEARCH,
+VCORES,
+PHYSICALMEMORYBYTES,
+NETWORKINFO,
+HOSTNAME,
+IPADDRESS,
+AGENTINFO,
+AGENTCLASS,
+UPTIME,
+AGENTMANIFEST,
+AGENTTYPE,
+IOMANIFEST,
+PROPERTYDESCRIPTORS,
+ECUINFO,
+OPERATION,
+OPERAND,
+FILEPATH,
+CHUNKSIZEBYTES,
+DELIMITER,
+TAILFREQMILLISECONDS,
+TCPPORT,
+NIFIPORT,
+REQUESTEDOPERATIONS
+} c2_keys_t;
+
+value_t value_uint8(uint8_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint8 = value;
+val.val_type = UINT8_TYPE;
+return val;
+}
+
+value_t value_uint16(uint16_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint16 = value;
+val.val_type = UINT16_TYPE;
+return val;
+}
+
+value_t value_uint32(uint32_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint32 = value;
+val.val_type = UINT32_TYPE;
+return val;
+}
+
+value_t value_uint64(uint64_t value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_uint64 = value;
+val.val_type = UINT64_TYPE;
+return val;
+}
+
+value_t value_string(const char * value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+copystr(value, _str);
+val.val_type = STRING_TYPE;
+return val;
+}
+
+value_t value_nstring(const unsigned char * value, size_t len) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_str = (char *)malloc(len + 1);
+memset(val.v_str, 0, len + 1);
+memcpy(val.v_str, value, len);
+val.val_type = STRING_TYPE;
+return val;
+}
+
+value_t value_map(c2_payload_map_t * value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_map = value;
+val.val_type = HASH_TYPE;
+return val;
+}
+
+value_t value_list(c2_payload_list_t * value) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_maplist = value;
+val.val_type = LIST_TYPE;
+return val;
+}
+
+value_t value_property(properties_t * props) {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.v_props = clone_properties(props);
+val.val_type = PROP_TYPE;
+return val;
+}
+
+value_t value_none() {
+value_t val;
+memset(, 0, sizeof(value_t));
+val.val_type = NONE_TYPE;
+return val;
+}
+
+int is_value_none(value_t val) {
+return val.val_type == NONE_TYPE;
+}
+
+int is_value_uint(value_t val) {
+return val.val_type == UINT8_TYPE ||
+val.val_type == UINT16_TYPE ||
+val.val_type == UINT32_TYPE ||
+val.val_type == UINT64_TYPE;
+}
+
+int is_value_string(value_t val) {
+return val.val_type == STRING_TYPE;
+}
+
+int is_value_map(value_t val) {
+return val.val_type == HASH_TYPE;
+}
+
+int is_value_list(value_t val) {
+return val.val_type == LIST_TYPE;
+}
+
+int is_value_prop(value_t val) {
+return val.val_type == PROP_TYPE;
+}
+
+c2_payload_map_t * allocate_c2_payload() {
+c2_payload_map_t * map = (c2_payload_map_t
*)malloc(sizeof(c2_payload_map_t));
+memset(map, 0, sizeof(c2_payload_map_t));
+return map;
+}
+
+c2_payload_map_t * add_kvp(c2_payload_map_t ** map, uint16_t key, value_t
value) {
+c2_payload_map_t * field = allocate_c2_payload();
+field->key = key;
+field->value = value;
+HASH_ADD_INT((*map), key, field);
+return field;
+}
+
+c2_payload_list_t * add_list(c2_payload_list_t ** list, value_t value) {
+if (!list) return NULL;
+c2_payload_list_t * node = (c2_payload_list_t
*)malloc(sizeof(c2_payload_list_t));
+memset(node, 0, sizeof(c2_payload_list_t));
+node->value = value;
+LL_APPEND((*list), node);
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371958040
##
File path: nanofi/src/core/ring_buffer.c
##
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+#include
+
+size_t min(size_t x, size_t y) {
+if (x <= y) {
+return x;
+}
+return y;
+}
+
+size_t write_ring_buffer(ring_buffer_t * rb, const char * payload, size_t
length) {
+if (!rb || !rb->data || !payload || !length) {
+return 0;
+}
+
+if (rb->capacity - rb->size == 0) {
+//buffer is full
+return 0;
+}
+
+size_t capacity = rb->capacity;
+size_t available = capacity - rb->size;
+size_t bytes_to_write = min(available, length);
+
+if (bytes_to_write <= (capacity - rb->write_index)) {
+memcpy(rb->data + rb->write_index, payload, bytes_to_write);
+rb->write_index += bytes_to_write;
+if (rb->write_index == rb->capacity) {
+rb->write_index = 0;
+}
+} else {
+//rotate and write
+size_t size1 = capacity - rb->write_index;
+memcpy(rb->data + rb->write_index, payload, size1);
+size_t size2 = bytes_to_write - size1;
+memcpy(rb->data, payload + size1, size2);
+rb->write_index = size2;
+}
+rb->size += bytes_to_write;
+return bytes_to_write;
+}
+
+size_t read_ring_buffer(ring_buffer_t * rb, char * payload, size_t length) {
+if (!rb || !rb->data || !payload || !length) {
+return 0;
+}
+
+if (rb->size == 0) {
+//buffer is empty
+return 0;
+}
+
+size_t capacity = rb->capacity;
+size_t bytes_to_read = min(rb->size, length);
+
+if (bytes_to_read <= (capacity - rb->read_index)) {
+memcpy(payload, rb->data + rb->read_index, bytes_to_read);
+rb->read_index += bytes_to_read;
+if (rb->read_index == rb->capacity) {
+rb->read_index = 0;
+}
+} else {
+size_t size1 = capacity - rb->read_index;
+memcpy(payload, rb->data + rb->read_index, size1);
+size_t size2 = bytes_to_read - size1;
+memcpy(payload + size1, rb->data, size2);
+rb->read_index = size2;
+}
+rb->size -= bytes_to_read;
+return bytes_to_read;
+}
+
+size_t check_available_space(ring_buffer_t * rb) {
+//this is a hypothetical situation. no harm in safety check
+if (rb->capacity < rb->size)
+return 0;
+return rb->capacity - rb->size;
+}
+
+void free_ring_buffer(ring_buffer_t * rb) {
+free(rb->data);
Review comment:
Missing `#include `
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2 integration.
szaszm commented on a change in pull request #674: Minificpp 1007 - ECU C2
integration.
URL: https://github.com/apache/nifi-minifi-cpp/pull/674#discussion_r371795811
##
File path: nanofi/include/ecu_api/io_api.h
##
@@ -0,0 +1,111 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+#ifndef NANOFI_INCLUDE_ECU_API_IO_API_H_
+#define NANOFI_INCLUDE_ECU_API_IO_API_H_
+
+#include
+
+typedef enum io_type {
+TAILFILE,
+SITE2SITE,
+KAFKA,
+MQTTIO,
+MANUAL
+} io_type_t;
+
+static const char * io_type_str[MANUAL+1] = {"FILE", "SITE2SITE", "KAFKA",
"MQTT", "MANUAL"};
Review comment:
inconsistency between `TAILFILE`/`"FILE"` and `MQTTIO`/`"MQTT"`. Is it
intentional? If so, what's the intention? (Please describe in code comment.)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Resolved] (NIFI-7066) Default value documented for nifi.analytics.query.interval property is incorrect
[
https://issues.apache.org/jira/browse/NIFI-7066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew M. Lim resolved NIFI-7066.
-
Fix Version/s: 1.12.0
Resolution: Fixed
> Default value documented for nifi.analytics.query.interval property is
> incorrect
>
>
> Key: NIFI-7066
> URL: https://issues.apache.org/jira/browse/NIFI-7066
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Documentation Website
>Reporter: Andrew M. Lim
>Assignee: Andrew M. Lim
>Priority: Minor
> Fix For: 1.12.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The Admin Guide description for this property (found in
> [https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#analytics_properties)]
> states the default value for {{nifi.analytics.query.interval}} is {{3
> mins}}. But it is {{5 mins}} in the default {{nifi.properties}} file.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] arpadboda opened a new pull request #718: MINIFICPP-1132 - CWEL should stop committing further flowfiles in cas…
arpadboda opened a new pull request #718: MINIFICPP-1132 - CWEL should stop committing further flowfiles in cas… URL: https://github.com/apache/nifi-minifi-cpp/pull/718 …e outgoing connection is full Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (MINIFICPP-1132) CWEL should stop committing further flowfiles in case outgoing connection is full
Arpad Boda created MINIFICPP-1132: - Summary: CWEL should stop committing further flowfiles in case outgoing connection is full Key: MINIFICPP-1132 URL: https://issues.apache.org/jira/browse/MINIFICPP-1132 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Affects Versions: 0.7.0 Reporter: Arpad Boda Assignee: Arpad Boda Fix For: 0.8.0 The current logic of CWEL bypass backpressure limits. OnTrigger should return in case the outgoing connection is full. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (MINIFICPP-1131) MiNiFi cpp-0.7.0 installation on rhel7 failing
[ https://issues.apache.org/jira/browse/MINIFICPP-1131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz reassigned MINIFICPP-1131: --- Assignee: Marton Szasz > MiNiFi cpp-0.7.0 installation on rhel7 failing > --- > > Key: MINIFICPP-1131 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1131 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Affects Versions: 0.7.0 > Environment: MiNiFi++ cpp-0.7.0 > RHEL/CentOS 7 >Reporter: CHANDAN KUMAR >Assignee: Marton Szasz >Priority: Major > > # ./minifi.sh install dataflow > ./minifi.sh: line 120: % 5: syntax error: operand expected (error token is "% > 5") > Could not create service file /etc/init.d/dataflow > > I see few more lines of code added in minifi.sh script in cpp-0.7.0 as > compare to cpp-0.6.0 (Commenting out lines 169 to 279 seems to solve the > problem) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (MINIFICPP-1131) MiNiFi cpp-0.7.0 installation on rhel7 failing
[ https://issues.apache.org/jira/browse/MINIFICPP-1131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025235#comment-17025235 ] Marton Szasz edited comment on MINIFICPP-1131 at 1/28/20 3:53 PM: -- The changes to the installer script between 0.6.0 and 0.7.0 didn't properly escape bash variable references in the heredoc containing the init script. was (Author: szaszm): The issue was introduced in context of MINIFI-914. The changes didn't properly escape bash variable references in the heredoc containing the init script. > MiNiFi cpp-0.7.0 installation on rhel7 failing > --- > > Key: MINIFICPP-1131 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1131 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Affects Versions: 0.7.0 > Environment: MiNiFi++ cpp-0.7.0 > RHEL/CentOS 7 >Reporter: CHANDAN KUMAR >Priority: Major > > # ./minifi.sh install dataflow > ./minifi.sh: line 120: % 5: syntax error: operand expected (error token is "% > 5") > Could not create service file /etc/init.d/dataflow > > I see few more lines of code added in minifi.sh script in cpp-0.7.0 as > compare to cpp-0.6.0 (Commenting out lines 169 to 279 seems to solve the > problem) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (MINIFICPP-1131) MiNiFi cpp-0.7.0 installation on rhel7 failing
[ https://issues.apache.org/jira/browse/MINIFICPP-1131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025235#comment-17025235 ] Marton Szasz commented on MINIFICPP-1131: - The issue was introduced in context of MINIFI-914. The changes didn't properly escape bash variable references in the heredoc containing the init script. > MiNiFi cpp-0.7.0 installation on rhel7 failing > --- > > Key: MINIFICPP-1131 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1131 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Affects Versions: 0.7.0 > Environment: MiNiFi++ cpp-0.7.0 > RHEL/CentOS 7 >Reporter: CHANDAN KUMAR >Priority: Major > > # ./minifi.sh install dataflow > ./minifi.sh: line 120: % 5: syntax error: operand expected (error token is "% > 5") > Could not create service file /etc/init.d/dataflow > > I see few more lines of code added in minifi.sh script in cpp-0.7.0 as > compare to cpp-0.6.0 (Commenting out lines 169 to 279 seems to solve the > problem) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] bakaid commented on a change in pull request #715: MINIFICPP-1126 - Reduce sawtooth in memory usage of rocksdb flowfile …
bakaid commented on a change in pull request #715: MINIFICPP-1126 - Reduce sawtooth in memory usage of rocksdb flowfile … URL: https://github.com/apache/nifi-minifi-cpp/pull/715#discussion_r371838409 ## File path: extensions/rocksdb-repos/FlowFileRepository.h ## @@ -105,8 +105,15 @@ class FlowFileRepository : public core::Repository, public std::enable_shared_fr options.create_if_missing = true; options.use_direct_io_for_flush_and_compaction = true; options.use_direct_reads = true; + +// Write buffers are used as db oepration logs. When they get filled the events are merged and serialized. +// The default size is 64MB. +// In our case it's usually too much, causing sawtooth in memory consumption. (Consumes more than the whole MiniFi) +// To avoid DB write issues during heavy load it's recommended to have high number of buffer. +// Rocksdb's stall featur can also trigger in case the number of buffers is >= 3. Review comment: typo: featur -> feature This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-minifi-cpp] bakaid commented on a change in pull request #715: MINIFICPP-1126 - Reduce sawtooth in memory usage of rocksdb flowfile …
bakaid commented on a change in pull request #715: MINIFICPP-1126 - Reduce sawtooth in memory usage of rocksdb flowfile … URL: https://github.com/apache/nifi-minifi-cpp/pull/715#discussion_r371838161 ## File path: extensions/rocksdb-repos/FlowFileRepository.h ## @@ -105,8 +105,15 @@ class FlowFileRepository : public core::Repository, public std::enable_shared_fr options.create_if_missing = true; options.use_direct_io_for_flush_and_compaction = true; options.use_direct_reads = true; + +// Write buffers are used as db oepration logs. When they get filled the events are merged and serialized. Review comment: typo: oepration -> operation This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Deleted] (MINIFICPP-1130) Install of minifi as a service via minfi.sh does not complete successfully
[
https://issues.apache.org/jira/browse/MINIFICPP-1130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aldrin Piri deleted MINIFICPP-1130:
---
> Install of minifi as a service via minfi.sh does not complete successfully
> --
>
> Key: MINIFICPP-1130
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1130
> Project: Apache NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Aldrin Piri
>Priority: Major
>
> On RHEL/CentOS 7 trying to install leads to the following error:
> {quote}[root@4d66d22dfec0 nifi-minifi-cpp-0.7.0]# ./bin/minifi.sh install
> minifi
> ./bin/minifi.sh: line 120: % 5: syntax error: operand expected (error token
> is "% 5")
> Could not create service file /etc/init.d/minifi
> {quote}
> This can be recreated with the following, brute force, Docker-based one liner:
> {code:java}
> docker run -it centos:7 /bin/bash -c "yum install -y wget; cd; mkdir minifi:
> cd minifi; wget
> http://mirror.olnevhost.net/pub/apache/nifi/nifi-minifi-cpp/0.7.0/nifi-minifi-cpp-centos-0.7.0-bin.tar.gz;
> tar xf nifi-minifi-cpp-centos-0.7.0-bin.tar.gz; cd nifi-minifi-cpp-0.7.0/;
> ./bin/minifi.sh install minifi"{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (MINIFICPP-1131) MiNiFi cpp-0.7.0 installation on rhel7 failing
[
https://issues.apache.org/jira/browse/MINIFICPP-1131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025113#comment-17025113
]
Aldrin Piri commented on MINIFICPP-1131:
This can be recreated with the following, brute force, Docker-based one liner:
{code:java}
docker run -it centos:7 /bin/bash -c "yum install -y wget; cd; mkdir minifi: cd
minifi; wget
http://mirror.olnevhost.net/pub/apache/nifi/nifi-minifi-cpp/0.7.0/nifi-minifi-cpp-centos-0.7.0-bin.tar.gz;
tar xf nifi-minifi-cpp-centos-0.7.0-bin.tar.gz; cd nifi-minifi-cpp-0.7.0/;
./bin/minifi.sh install minifi"{code}
> MiNiFi cpp-0.7.0 installation on rhel7 failing
> ---
>
> Key: MINIFICPP-1131
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1131
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Affects Versions: 0.7.0
> Environment: MiNiFi++ cpp-0.7.0
> RHEL/CentOS 7
>Reporter: CHANDAN KUMAR
>Priority: Major
>
> # ./minifi.sh install dataflow
> ./minifi.sh: line 120: % 5: syntax error: operand expected (error token is "%
> 5")
> Could not create service file /etc/init.d/dataflow
>
> I see few more lines of code added in minifi.sh script in cpp-0.7.0 as
> compare to cpp-0.6.0 (Commenting out lines 169 to 279 seems to solve the
> problem)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Moved] (MINIFICPP-1131) MiNiFi cpp-0.7.0 installation on rhel7 failing
[ https://issues.apache.org/jira/browse/MINIFICPP-1131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aldrin Piri moved NIFI-7071 to MINIFICPP-1131: -- Component/s: (was: Configuration) Key: MINIFICPP-1131 (was: NIFI-7071) Affects Version/s: (was: 0.7.0) 0.7.0 Project: Apache NiFi MiNiFi C++ (was: Apache NiFi) > MiNiFi cpp-0.7.0 installation on rhel7 failing > --- > > Key: MINIFICPP-1131 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1131 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Affects Versions: 0.7.0 > Environment: MiNiFi++ cpp-0.7.0 > RHEL/CentOS 7 >Reporter: CHANDAN KUMAR >Priority: Major > Labels: triage > > # ./minifi.sh install dataflow > ./minifi.sh: line 120: % 5: syntax error: operand expected (error token is "% > 5") > Could not create service file /etc/init.d/dataflow > > I see few more lines of code added in minifi.sh script in cpp-0.7.0 as > compare to cpp-0.6.0 (Commenting out lines 169 to 279 seems to solve the > problem) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1131) MiNiFi cpp-0.7.0 installation on rhel7 failing
[ https://issues.apache.org/jira/browse/MINIFICPP-1131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aldrin Piri updated MINIFICPP-1131: --- Labels: (was: triage) > MiNiFi cpp-0.7.0 installation on rhel7 failing > --- > > Key: MINIFICPP-1131 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1131 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Affects Versions: 0.7.0 > Environment: MiNiFi++ cpp-0.7.0 > RHEL/CentOS 7 >Reporter: CHANDAN KUMAR >Priority: Major > > # ./minifi.sh install dataflow > ./minifi.sh: line 120: % 5: syntax error: operand expected (error token is "% > 5") > Could not create service file /etc/init.d/dataflow > > I see few more lines of code added in minifi.sh script in cpp-0.7.0 as > compare to cpp-0.6.0 (Commenting out lines 169 to 279 seems to solve the > problem) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (MINIFICPP-1130) Install of minifi as a service via minfi.sh does not complete successfully
Aldrin Piri created MINIFICPP-1130:
--
Summary: Install of minifi as a service via minfi.sh does not
complete successfully
Key: MINIFICPP-1130
URL: https://issues.apache.org/jira/browse/MINIFICPP-1130
Project: Apache NiFi MiNiFi C++
Issue Type: Improvement
Affects Versions: 0.7.0
Reporter: Aldrin Piri
On RHEL/CentOS 7 trying to install leads to the following error:
{quote}[root@4d66d22dfec0 nifi-minifi-cpp-0.7.0]# ./bin/minifi.sh install minifi
./bin/minifi.sh: line 120: % 5: syntax error: operand expected (error token is
"% 5")
Could not create service file /etc/init.d/minifi
{quote}
This can be recreated with the following, brute force, Docker-based one liner:
{code:java}
docker run -it centos:7 /bin/bash -c "yum install -y wget; cd; mkdir minifi: cd
minifi; wget
http://mirror.olnevhost.net/pub/apache/nifi/nifi-minifi-cpp/0.7.0/nifi-minifi-cpp-centos-0.7.0-bin.tar.gz;
tar xf nifi-minifi-cpp-centos-0.7.0-bin.tar.gz; cd nifi-minifi-cpp-0.7.0/;
./bin/minifi.sh install minifi"{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (NIFI-7071) MiNiFi cpp-0.7.0 installation on rhel7 failing
CHANDAN KUMAR created NIFI-7071: --- Summary: MiNiFi cpp-0.7.0 installation on rhel7 failing Key: NIFI-7071 URL: https://issues.apache.org/jira/browse/NIFI-7071 Project: Apache NiFi Issue Type: Bug Components: Configuration Affects Versions: 0.7.0 Environment: MiNiFi++ cpp-0.7.0 RHEL/CentOS 7 Reporter: CHANDAN KUMAR # ./minifi.sh install dataflow ./minifi.sh: line 120: % 5: syntax error: operand expected (error token is "% 5") Could not create service file /etc/init.d/dataflow I see few more lines of code added in minifi.sh script in cpp-0.7.0 as compare to cpp-0.6.0 (Commenting out lines 169 to 279 seems to solve the problem) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] woutifier-t commented on issue #3977: NIFI-7007 Add update functionality to the PutCassandraRecord processor.
woutifier-t commented on issue #3977: NIFI-7007 Add update functionality to the PutCassandraRecord processor. URL: https://github.com/apache/nifi/pull/3977#issuecomment-579218673 > The `PutCassandraRecordUpdateTest.java` needs a license but otherwise this looks ready. Done :-). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (MINIFICPP-1129) Clean up repo size property of repositories
Arpad Boda created MINIFICPP-1129: - Summary: Clean up repo size property of repositories Key: MINIFICPP-1129 URL: https://issues.apache.org/jira/browse/MINIFICPP-1129 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Affects Versions: 0.7.0 Reporter: Arpad Boda Fix For: 1.0.0 There is a member in the base class and there are getters, but the counters in different repos are badly maintained. I think it's not used via the base class, but the results won't be too fruitful anyway, so I think we could completely remove this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] arpadboda commented on a change in pull request #715: MINIFICPP-1126 - Reduce sawtooth in memory usage of rocksdb flowfile …
arpadboda commented on a change in pull request #715: MINIFICPP-1126 - Reduce
sawtooth in memory usage of rocksdb flowfile …
URL: https://github.com/apache/nifi-minifi-cpp/pull/715#discussion_r371749842
##
File path: extensions/rocksdb-repos/FlowFileRepository.cpp
##
@@ -69,14 +68,8 @@ void FlowFileRepository::flush() {
batch.Delete(keys[i]);
}
-
- if (db_->Write(rocksdb::WriteOptions(), ).ok()) {
-logger_->log_trace("Decrementing %u from a repo size of %u",
decrement_total, repo_size_.load());
-if (decrement_total > repo_size_.load()) {
- repo_size_ = 0;
-} else {
- repo_size_ -= decrement_total;
-}
+ if (!db_->Write(rocksdb::WriteOptions(), ).ok()) {
+logger_->log_warn("Failed to execute batch operation when flushing
FlowFileRepository");
Review comment:
https://issues.apache.org/jira/browse/MINIFICPP-1129
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] arpadboda commented on a change in pull request #715: MINIFICPP-1126 - Reduce sawtooth in memory usage of rocksdb flowfile …
arpadboda commented on a change in pull request #715: MINIFICPP-1126 - Reduce
sawtooth in memory usage of rocksdb flowfile …
URL: https://github.com/apache/nifi-minifi-cpp/pull/715#discussion_r371745162
##
File path: extensions/rocksdb-repos/FlowFileRepository.cpp
##
@@ -89,23 +82,28 @@ void FlowFileRepository::flush() {
}
}
-void FlowFileRepository::run() {
- // threshold for purge
+void FlowFileRepository::printStats() {
Review comment:
Fair point.
I would keep it at info level, but call less frequently.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] bakaid commented on a change in pull request #715: MINIFICPP-1126 - Reduce sawtooth in memory usage of rocksdb flowfile …
bakaid commented on a change in pull request #715: MINIFICPP-1126 - Reduce sawtooth in memory usage of rocksdb flowfile … URL: https://github.com/apache/nifi-minifi-cpp/pull/715#discussion_r371702847 ## File path: extensions/rocksdb-repos/FlowFileRepository.h ## @@ -103,6 +105,9 @@ class FlowFileRepository : public core::Repository, public std::enable_shared_fr options.create_if_missing = true; options.use_direct_io_for_flush_and_compaction = true; options.use_direct_reads = true; +options.write_buffer_size = 8 << 20; Review comment: I would like to see more in-code comments about the rationale behind these values. Why 8 MB? Why do we set max_write_buffer_number to 4? As far as I understand that means it can keep 4 of these memtables in memory, and the default is 2. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-minifi-cpp] bakaid commented on a change in pull request #715: MINIFICPP-1126 - Reduce sawtooth in memory usage of rocksdb flowfile …
bakaid commented on a change in pull request #715: MINIFICPP-1126 - Reduce
sawtooth in memory usage of rocksdb flowfile …
URL: https://github.com/apache/nifi-minifi-cpp/pull/715#discussion_r371703217
##
File path: extensions/rocksdb-repos/FlowFileRepository.cpp
##
@@ -69,14 +68,8 @@ void FlowFileRepository::flush() {
batch.Delete(keys[i]);
}
-
- if (db_->Write(rocksdb::WriteOptions(), ).ok()) {
-logger_->log_trace("Decrementing %u from a repo size of %u",
decrement_total, repo_size_.load());
-if (decrement_total > repo_size_.load()) {
- repo_size_ = 0;
-} else {
- repo_size_ -= decrement_total;
-}
+ if (!db_->Write(rocksdb::WriteOptions(), ).ok()) {
+logger_->log_warn("Failed to execute batch operation when flushing
FlowFileRepository");
Review comment:
Agreed, but I think it is something we would need, done properly. Is there a
follow-up issues for it?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] bakaid commented on a change in pull request #715: MINIFICPP-1126 - Reduce sawtooth in memory usage of rocksdb flowfile …
bakaid commented on a change in pull request #715: MINIFICPP-1126 - Reduce
sawtooth in memory usage of rocksdb flowfile …
URL: https://github.com/apache/nifi-minifi-cpp/pull/715#discussion_r371693362
##
File path: extensions/rocksdb-repos/FlowFileRepository.cpp
##
@@ -89,23 +82,28 @@ void FlowFileRepository::flush() {
}
}
-void FlowFileRepository::run() {
- // threshold for purge
+void FlowFileRepository::printStats() {
Review comment:
If I understand correctly, this is ran every purge_period, which is, by
default, 2.5 s.
4 log lines every 2.5 seconds is way too much noise, affecting everyone, as
virtually everyone uses this FlowFileRepo and the default log level is info.
This would account for the majority of logs in many cases.
I think this should definitely be at debug level.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (NIFI-7012) Using sensitive parameter in sensitive property of InvokeScriptedProcessor causes Jetty shutdown on NiFi restart
[
https://issues.apache.org/jira/browse/NIFI-7012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024992#comment-17024992
]
Dariusz Chmielewski commented on NIFI-7012:
---
Thank you [~alopresto]! This patch does address the NiFi startup issue, but
while I was testing this I did notice one side issue. If you try to change
sensitive parameter value while InvokeScriptedProcessor using it is currently
running you will get below exception and parameter value does not change. You
have to manually stop the processor first to successfully change the sensitive
parameter value.
{code:java}
2020-01-28 09:12:26,686 INFO [Parameter Context Update Thread-1]
o.a.n.web.api.ParameterContextResource Disabling 0 Controller Services in order
to update Parameter Context2020-01-28 09:12:26,686 INFO [Parameter Context
Update Thread-1] o.a.n.web.api.ParameterContextResource Disabling 0 Controller
Services in order to update Parameter Context2020-01-28 09:12:26,687 INFO
[Parameter Context Update Thread-1] o.a.n.web.api.ParameterContextResource
Updating Parameter Context with ID
900a6ced-016f-1000-13a0-956c5fa9adec2020-01-28 09:12:26,687 INFO [Parameter
Context Update Thread-1] o.a.n.web.api.ParameterContextResource Re-Enabling 0
Controller Services after having updated Parameter Context2020-01-28
09:12:26,687 INFO [Parameter Context Update Thread-1]
o.a.n.web.api.ParameterContextResource Restarting 1 Processors after having
updated Parameter Context2020-01-28 09:12:26,914 INFO [Flow Service Tasks
Thread-2] o.a.nifi.controller.StandardFlowService Saved flow controller
org.apache.nifi.controller.FlowController@327a1949 // Another save pending =
false2020-01-28 09:12:27,349 INFO [Parameter Context Update Thread-1]
o.a.n.c.s.StandardProcessScheduler Starting
InvokeScriptedProcessor[id=90151674-016f-1000-9378-1054622ee2a1]2020-01-28
09:12:27,349 INFO [Parameter Context Update Thread-1]
o.a.n.controller.StandardProcessorNode Starting
InvokeScriptedProcessor[id=90151674-016f-1000-9378-1054622ee2a1]2020-01-28
09:12:27,562 INFO [Timer-Driven Process Thread-4]
o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled
InvokeScriptedProcessor[id=90151674-016f-1000-9378-1054622ee2a1] to run with 1
threads2020-01-28 09:12:27,765 INFO [Flow Service Tasks Thread-2]
o.a.nifi.controller.StandardFlowService Saved flow controller
org.apache.nifi.controller.FlowController@327a1949 // Another save pending =
false2020-01-28 09:12:28,058 ERROR [Parameter Context Update Thread-1]
o.a.n.web.api.ParameterContextResource Failed to update Parameter
Contextjava.lang.IllegalStateException: Cannot update Parameter Context Test
because the update would add a Sensitive Parameter named 'test_pass' but this
Parameter already is referenced by a Non-Sensitive Property. at
org.apache.nifi.web.dao.impl.StandardParameterContextDAO.verifyParameterUpdate(StandardParameterContextDAO.java:189)
at
org.apache.nifi.web.dao.impl.StandardParameterContextDAO.verifyUpdate(StandardParameterContextDAO.java:153)
at
org.apache.nifi.web.dao.impl.StandardParameterContextDAO$$FastClassBySpringCGLIB$$7099e091.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
at
org.apache.nifi.web.dao.impl.StandardParameterContextDAO$$EnhancerBySpringCGLIB$$90f1d6e7.verifyUpdate()
at
org.apache.nifi.web.StandardNiFiServiceFacade.verifyUpdateParameterContext(StandardNiFiServiceFacade.java:1004)
at
org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:84)
at
org.apache.nifi.web.NiFiServiceFacadeLock.proceedWithReadLock(NiFiServiceFacadeLock.java:161)
at
org.apache.nifi.web.NiFiServiceFacadeLock.verifyLock(NiFiServiceFacadeLock.java:138)
at sun.reflect.GeneratedMethodAccessor335.invoke(Unknown Source) at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
