[ 
https://issues.apache.org/jira/browse/NIFI-16583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18145284#comment-18145284
 ] 

Simon Theodosius commented on NIFI-16583:
-----------------------------------------

The standalone implementation now uses the ConsumeOPCUA name, and the latest 
fixes have passed GitHub CI.

Source commit:
https://github.com/paf91/nifi-opcua/commit/9e14293fabea1d2f6431860d5e8a1ae40aabb469

Complete implementation at that revision:
https://github.com/paf91/nifi-opcua/tree/9e14293fabea1d2f6431860d5e8a1ae40aabb469

Successful CI run:
https://github.com/paf91/nifi-opcua/actions/runs/43740998153

Integration into the Apache NiFi source tree remains to be completed. 
Maintainer feedback on the proposed output interface and Controller Service 
design would help guide that adaptation.

> Add ConsumeOPCUA processor for OPC UA data-change subscriptions
> ---------------------------------------------------------------
>
>                 Key: NIFI-16583
>                 URL: https://issues.apache.org/jira/browse/NIFI-16583
>             Project: Apache NiFi
>          Issue Type: New Feature
>          Components: Extensions
>            Reporter: Simon Theodosius
>            Priority: Major
>              Labels: opcua
>
> h3. Motivation
> OPC UA servers expose industrial measurements and their quality and 
> timestamps. A NiFi source processor that subscribes to value changes would 
> allow these measurements to enter a dataflow directly, with configurable 
> sampling and batching.
> I propose adding ConsumeOPCUA using Eclipse Milo and adapting an existing 
> standalone implementation to the current Apache NiFi codebase.
> h3. Proposed scope
> * Subscribe to OPC UA Value data changes over opc.tcp and emit initial values 
> and subsequent changes as batched FlowFiles.
> * Configure tags using namespace-index or namespace-URI NodeIds, with 
> aliases, sampling intervals, server queue sizes and deadbands.
> * Include tag identity, value, status and timestamps in the output.
> * Support anonymous and username/password authentication, explicit security 
> policy and message security mode selection, and client application 
> certificates. Full server certificate validation should be the default.
> * Support connection verification, reconnection and subscription recreation, 
> with primary-node execution in clustered NiFi.
> OPC UA Events/alarms, historical reads, writes and durable recovery are 
> outside the proposed initial scope.
> h3. Delivery and resource limits
> Bound retained notifications by count and serialized bytes. Keep batches 
> reserved and counted against both limits until NiFi reports the commit 
> outcome, and report buffer overflow and server-reported notification loss. 
> These limits do not bound all decoded protocol objects or total JVM heap.
> Shutdown should stop accepting notifications and make a time-limited attempt 
> to drain buffered data. This does not impose a deadline on NiFi repository 
> operations or the overall processor stop.
> Uncommitted notifications remain in memory and are not replicated. Outages, 
> overflow, JVM termination and primary-node failover can cause gaps or 
> duplicates. The processor would not provide end-to-end exactly-once or 
> lossless outage recovery.
> h3. Existing implementation and design questions
> The standalone implementation is available at 
> https://github.com/paf91/nifi-opcua and is currently published under the 
> processor name SubscribeOpcUa. ConsumeOPCUA is the proposed upstream name. 
> Integration into the Apache NiFi source tree and dependency/licensing review 
> remain to be completed.
> I would appreciate maintainer guidance on two initial design choices:
> * Start with JSON/NDJSON output, or expose a RecordSetWriterFactory in the 
> first contribution?
> * Keep OPC UA identity/trust configuration on the processor, or introduce an 
> OPC UA-specific Controller Service?
> h3. Proposed verification
> * Exercise supported authentication/security combinations and certificate 
> failures.
> * Cover namespace resolution, value encoding, reconnect/resubscribe, buffer 
> saturation, downstream backpressure, commit failures and Stop/Start races.
> * Measure memory and CPU use under representative load and repeated 
> connection failures.
> * Verify interoperability with an independent OPC UA server such as open62541 
> and load the resulting NAR in NiFi.
> * Complete the current NiFi contribution checks, dependency/licensing review 
> and processor documentation.
> h3. Related work
> * MINIFICPP-819 - OPC Unified Architecture Support.
> * MINIFICPP-893 - OPC UA Client Implementation.
> * MINIFICPP-2908 - Add processor to fetch OPC events.
> These concern MiNiFi C++. This proposal targets the Java NiFi runtime and 
> Value data-change subscriptions.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to