[
https://issues.apache.org/jira/browse/NIFI-16583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18145284#comment-18145284
]
Simon Theodosius edited comment on NIFI-16583 at 11/15/26, 6:03 PM:
--------------------------------------------------------------------
I have pushed the updated standalone ConsumeOPCUA implementation. GitHub CI
passed for commit 9e14293.
Source at this revision:
https://github.com/paf91/nifi-opcua/tree/9e14293fabea1d2f6431860d5e8a1ae40aabb469
Commit:
https://github.com/paf91/nifi-opcua/commit/9e14293fabea1d2f6431860d5e8a1ae40aabb469
CI results:
https://github.com/paf91/nifi-opcua/actions/runs/43740998153
Integration into the Apache NiFi source tree and full runtime acceptance of the
latest NARs remain outstanding.
For the upstream adaptation, I would appreciate guidance on two design choices:
- Should the initial contribution use RecordSetWriterFactory, or is the
existing JSON/NDJSON output suitable?
- Should client identity and server trust be configured on the processor
initially, or through an OPC UA-specific Controller Service?
was (Author: JIRAUSER313362):
The standalone implementation now uses the ConsumeOPCUA name, and the latest
fixes have passed GitHub CI.
Source commit:
https://github.com/paf91/nifi-opcua/commit/9e14293fabea1d2f6431860d5e8a1ae40aabb469
Complete implementation at that revision:
https://github.com/paf91/nifi-opcua/tree/9e14293fabea1d2f6431860d5e8a1ae40aabb469
Successful CI run:
https://github.com/paf91/nifi-opcua/actions/runs/43740998153
Integration into the Apache NiFi source tree remains to be completed.
Maintainer feedback on the proposed output interface and Controller Service
design would help guide that adaptation.
> Add ConsumeOPCUA processor for OPC UA data-change subscriptions
> ---------------------------------------------------------------
>
> Key: NIFI-16583
> URL: https://issues.apache.org/jira/browse/NIFI-16583
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
> Reporter: Simon Theodosius
> Priority: Major
> Labels: opcua
>
> h3. Motivation
> OPC UA servers expose industrial measurements and their quality and
> timestamps. A NiFi source processor that subscribes to value changes would
> allow these measurements to enter a dataflow directly, with configurable
> sampling and batching.
> I propose adding ConsumeOPCUA using Eclipse Milo and adapting an existing
> standalone implementation to the current Apache NiFi codebase.
> h3. Proposed scope
> * Subscribe to OPC UA Value data changes over opc.tcp and emit initial values
> and subsequent changes as batched FlowFiles.
> * Configure tags using namespace-index or namespace-URI NodeIds, with
> aliases, sampling intervals, server queue sizes and deadbands.
> * Include tag identity, value, status and timestamps in the output.
> * Support anonymous and username/password authentication, explicit security
> policy and message security mode selection, and client application
> certificates. Full server certificate validation should be the default.
> * Support connection verification, reconnection and subscription recreation,
> with primary-node execution in clustered NiFi.
> OPC UA Events/alarms, historical reads, writes and durable recovery are
> outside the proposed initial scope.
> h3. Delivery and resource limits
> Bound retained notifications by count and serialized bytes. Keep batches
> reserved and counted against both limits until NiFi reports the commit
> outcome, and report buffer overflow and server-reported notification loss.
> These limits do not bound all decoded protocol objects or total JVM heap.
> Shutdown should stop accepting notifications and make a time-limited attempt
> to drain buffered data. This does not impose a deadline on NiFi repository
> operations or the overall processor stop.
> Uncommitted notifications remain in memory and are not replicated. Outages,
> overflow, JVM termination and primary-node failover can cause gaps or
> duplicates. The processor would not provide end-to-end exactly-once or
> lossless outage recovery.
> h3. Existing implementation and design questions
> The standalone implementation is available at
> https://github.com/paf91/nifi-opcua and is currently published under the
> processor name SubscribeOpcUa. ConsumeOPCUA is the proposed upstream name.
> Integration into the Apache NiFi source tree and dependency/licensing review
> remain to be completed.
> I would appreciate maintainer guidance on two initial design choices:
> * Start with JSON/NDJSON output, or expose a RecordSetWriterFactory in the
> first contribution?
> * Keep OPC UA identity/trust configuration on the processor, or introduce an
> OPC UA-specific Controller Service?
> h3. Proposed verification
> * Exercise supported authentication/security combinations and certificate
> failures.
> * Cover namespace resolution, value encoding, reconnect/resubscribe, buffer
> saturation, downstream backpressure, commit failures and Stop/Start races.
> * Measure memory and CPU use under representative load and repeated
> connection failures.
> * Verify interoperability with an independent OPC UA server such as open62541
> and load the resulting NAR in NiFi.
> * Complete the current NiFi contribution checks, dependency/licensing review
> and processor documentation.
> h3. Related work
> * MINIFICPP-819 - OPC Unified Architecture Support.
> * MINIFICPP-893 - OPC UA Client Implementation.
> * MINIFICPP-2908 - Add processor to fetch OPC events.
> These concern MiNiFi C++. This proposal targets the Java NiFi runtime and
> Value data-change subscriptions.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)