Mark Payne created NIFI-3975:
--------------------------------
Summary: TailFile stopped tailing file
Key: NIFI-3975
URL: https://issues.apache.org/jira/browse/NIFI-3975
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Affects Versions: 1.2.0
Environment: Linux
Reporter: Mark Payne
I have a TailFile processor running on a 3-node cluster. It is tailing
logs/nifi-app.log. Two of the nodes seem to be working well. The third node is
not picking up any data. I enabled DEBUG logging and see the following logs:
{code}
2017-05-25 13:00:32,563 DEBUG [Timer-Driven Process Thread-48]
o.a.nifi.processors.standard.TailFile
TailFile[id=47f4a068-015a-1000-81b7-7d991bbbf80f] Recovering Rolled Off Files;
total number of files rolled off = 0
2017-05-25 13:00:32,563 DEBUG [Timer-Driven Process Thread-48]
o.a.nifi.processors.standard.TailFile
TailFile[id=47f4a068-015a-1000-81b7-7d991bbbf80f] Reading lines starting at
position 105133313
2017-05-25 13:00:32,563 DEBUG [Timer-Driven Process Thread-48]
o.a.nifi.processors.standard.TailFile
TailFile[id=47f4a068-015a-1000-81b7-7d991bbbf80f] No data to consume; removed
created FlowFile
2017-05-25 13:00:32,563 DEBUG [Timer-Driven Process Thread-48]
o.a.nifi.processors.standard.TailFile
TailFile[id=47f4a068-015a-1000-81b7-7d991bbbf80f] Returning LOCAL State:
StandardStateMap[version=442100928, values={file.0.length=0,
file.0.position=105133313, file.0.checksum=0,
file.0.filename=logs/nifi-app.log, file.0.timestamp=1495637757000}]
2017-05-25 13:00:32,563 DEBUG [Timer-Driven Process Thread-48]
o.a.nifi.processors.standard.TailFile
TailFile[id=47f4a068-015a-1000-81b7-7d991bbbf80f] Setting LOCAL State to
{file.0.length=0, file.0.position=105133313, file.0.checksum=0,
file.0.filename=logs/nifi-app.log, file.0.timestamp=1495637757000}
2017-05-25 13:00:32,563 DEBUG [Timer-Driven Process Thread-10]
o.a.nifi.processors.standard.TailFile
TailFile[id=47f4a068-015a-1000-81b7-7d991bbbf80f] Recovering Rolled Off Files;
total number of files rolled off = 0
2017-05-25 13:00:32,563 DEBUG [Timer-Driven Process Thread-10]
o.a.nifi.processors.standard.TailFile
TailFile[id=47f4a068-015a-1000-81b7-7d991bbbf80f] Reading lines starting at
position 105133313
2017-05-25 13:00:32,563 DEBUG [Timer-Driven Process Thread-10]
o.a.nifi.processors.standard.TailFile
TailFile[id=47f4a068-015a-1000-81b7-7d991bbbf80f] No data to consume; removed
created FlowFile
2017-05-25 13:00:32,563 DEBUG [Timer-Driven Process Thread-10]
o.a.nifi.processors.standard.TailFile
TailFile[id=47f4a068-015a-1000-81b7-7d991bbbf80f] Returning LOCAL State:
StandardStateMap[version=442100929, values={file.0.length=0,
file.0.position=105133313, file.0.checksum=0,
file.0.filename=logs/nifi-app.log, file.0.timestamp=1495637757000}]
2017-05-25 13:00:32,563 DEBUG [Timer-Driven Process Thread-10]
o.a.nifi.processors.standard.TailFile
TailFile[id=47f4a068-015a-1000-81b7-7d991bbbf80f] Setting LOCAL State to
{file.0.length=0, file.0.position=105133313, file.0.checksum=0,
file.0.filename=logs/nifi-app.log, file.0.timestamp=1495637757000}
2017-05-25 13:00:32,563 DEBUG [Timer-Driven Process Thread-10]
o.a.nifi.processors.standard.TailFile
TailFile[id=47f4a068-015a-1000-81b7-7d991bbbf80f] Recovering Rolled Off Files;
total number of files rolled off = 0
2017-05-25 13:00:32,563 DEBUG [Timer-Driven Process Thread-10]
o.a.nifi.processors.standard.TailFile
TailFile[id=47f4a068-015a-1000-81b7-7d991bbbf80f] Reading lines starting at
position 105133313
2017-05-25 13:00:32,563 DEBUG [Timer-Driven Process Thread-10]
o.a.nifi.processors.standard.TailFile
TailFile[id=47f4a068-015a-1000-81b7-7d991bbbf80f] No data to consume; removed
created FlowFile
2017-05-25 13:00:32,563 DEBUG [Timer-Driven Process Thread-10]
o.a.nifi.processors.standard.TailFile
TailFile[id=47f4a068-015a-1000-81b7-7d991bbbf80f] Returning LOCAL State:
StandardStateMap[version=442100930, values={file.0.length=0,
file.0.position=105133313, file.0.checksum=0,
file.0.filename=logs/nifi-app.log, file.0.timestamp=1495637757000}]
{code}
The file is approximately 33 MB right now. But the state indicates that the
position it is at in the file is over 105 MB; also, according to state, the
checksum is 0 and the file length is 0. This likely indicates that the file was
about 105 MB at one point and then it rolled over and stopped processing
properly, given that it thinks the checksum is 0.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
