[
https://issues.apache.org/jira/browse/NIFI-4169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16088077#comment-16088077
]
Y Wikander commented on NIFI-4169:
----------------------------------
After reviewing
[this|https://issues.apache.org/jira/browse/NIFI-4169?focusedCommentId=16083331&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16083331]
and
[this|https://issues.apache.org/jira/browse/NIFI-4169?focusedCommentId=16083547&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16083547]
comment, here's my current thoughts...
PutWebSocket needs to support the following types of inbound flowfiles:
# ListenWebSocket, non-broadcast (has websocket.session.id attribute)
# ListenWebSocket, broadcast (no websocket.session.id attribute); broadcast
list can be more than 1
# ConnectWebSocket, broadcast (no websocket.session.id attribute); broadcast
will be up to 1
# 1 and ( 2 or 3 )
I think that only #3 - the one I'm interested in - is the one that cannot
afford data loss (because NiFi is the source of the data). The theory is, that
since the others asked for the data, they are expecting it. So if they don't
get it they will ask for it again.
#3 can't afford data loss because the endpoint is being pushed to.
To accomplish that, consider the following logic:
- if no websocket.session.id attribute
-- Transfer the incoming FlowFile to success if it's sent to at least one
target.
-- Transfer the incoming FlowFile to failure if it's not send to any target.
-- Continue looping through sessions even if one throws Exception
-- Catch Exception while in the loop, then generate (e.g. clone) FlowFile to
send to 'failure' relationship. Embed failed websocket session id to
'websocket.session.id' FlowFile attribute. Also add attribute
'websocket.broadcast'.
- else
-- Try to transfer the incoming FlowFile
--- if attribute 'websocket.broadcast' exists and ( success or
session.unknown ) then terminate relationship (on the FlowFile)
--- transfer to session.unknown, success or failure (based on results)
- end
The purpose of attribute 'websocket.broadcast' it to catch the clones; because
if you don't they will transfer to success -- but we don't want that because
the original flowfile has already been send down the success road (earlier).
While not necessary for what I'm trying to accomplish...
Another relationship would be added called 'session unknown'. It represents
that the session id no longer exists. This would allow #1 and #2 usages to
check Automatically Terminate Relationship. In this way those usages can throw
the flowfiles away easily vs. trying to process failure reasons which would
otherwise loop in failure forever.
> PutWebSocket processor with blank WebSocket session id attribute cannot
> transfer to failure queue
> -------------------------------------------------------------------------------------------------
>
> Key: NIFI-4169
> URL: https://issues.apache.org/jira/browse/NIFI-4169
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Affects Versions: 1.3.0
> Reporter: Y Wikander
> Priority: Critical
> Labels: patch
> Attachments:
> 0001-websocket-when-sendMessage-fails-under-blank-session.patch
>
>
> If a PutWebSocket processor is setup with a blank WebSocket session id
> attribute (see NIFI-3318; Send message from PutWebSocket to all connected
> clients) and it is not connected to a websocket server it will log the
> failure and mark the flowfile with Success (rather than Failure) -- and the
> data is effectively lost.
> If there are multiple connected clients, and some succeed and others fail,
> routing Failure back into the PutWebSocket could result in duplicate data to
> some clients.
> Other NiFi processors seem to err on the side of "at least once".
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
