All, Still working on the lumberjack processor. Data is currently being decoded, SSL is sort of working but before I start wrapping up I wanted to confirm:
Lumberjack is a protocol that includes the dispatch of an acknowledgement message to the producing agent. As consequence, usually a producer tailing a file will only update its offset AFTER receiving the acknowledgement from the lumberjack endpoint. Ideally this acknowledgement should only be sent after data is no longer in the processor memory buffers and the chances of memory loss are restricted to catastrophic failure. Which leads to my question: From a development point of view, at what stage data is assumed to be under NiFi's care? I thank you in advance.
