I could if the Nifi was up and running but when I loose the node, the Nifi process itself dies so I can hardly take a heap dump of something that is not running 😉
It happened twice before I sent my first email...nothing since then. ________________________________ From: DEHAY Aurelien <[email protected]> Sent: Wednesday, September 25, 2019 10:29 AM To: [email protected] <[email protected]> Subject: RE: node died unexpectedly Classification\_- INTERNAL & PARTNERS Hello. By any chance, did you take thread dumps? If it happens again, take thread dumps of the java nifi process (with jstack: jstack -l <pid> >> threaddumps.log). Take 3 or 4 every 10/15 seconds . Regards. From: Jean-Sebastien Vachon <[email protected]> Sent: lundi 23 septembre 2019 15:08 To: [email protected] Subject: node died unexpectedly Hi, I am running a cluster of five nodes and one of them just quit for no apparent reason... at least I could not find anything wrong on the system. Here is a section of the logs. You can clearly see that there is a 8 minutes gap before I manually had to restart Nifi. I've looked in /var/log/messages and Nifi's log files but could not find any reason for this to happen... any thoughts/ideas? ----------- 2019-09-23 12:51:54,567 ERROR [Timer-Driven Process Thread-40] o.a.n.p.standard.ExecuteStreamCommand ExecuteStreamCommand[id=8b748c9a-7d7d-30cd-86a3-4ea3c4ccd3b4] Transferring flow file StandardFlowFileRecord[uuid=60d29ce2-088a-4829-a0ae-17672f1d86e6,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id =1569243036019-4497, container=default, section=401], offset=591157, length=-1],offset=0,name=186373,size=0] to nonzero status. Executable command /usr/bin/python3 ended in an error: 2019-09-23 12:51:54,570 ERROR [Timer-Driven Process Thread-4] o.a.n.p.standard.ExecuteStreamCommand ExecuteStreamCommand[id=272623bc-a371-3ff9-adaf-d8363b91a5ed] Transferring flow file StandardFlowFileRecord[uuid=a18485c8-9b03-43a8-8c13-e76803df9290,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1569242553862-4458, container=default, section=362], offset=780733, length=21229],offset=0,name=33287,size=21229] to nonzero status. Executable command /usr/bin/python3 ended in an error: /usr/local/lib/python3.7/site-packages/requests/__init__.py:80: RequestsDependencyWarning: urllib3 (1.23) or chardet (3.0.4) doesn't match a supported version! RequestsDependencyWarning) 2019-09-23 12:51:54,570 ERROR [Thread-130947] o.a.n.p.standard.ExecuteStreamCommand ExecuteStreamCommand[id=a7cb604a-465d-3440-beab-b9554e1ba4bb] Failed to write flow file to stdin due to java.io.IOException: Broken pipe: java.io.IOException: Broken pipe java.io.IOException: Broken pipe at java.io.FileOutputStream.writeBytes(Native Method) at java.io.FileOutputStream.write(FileOutputStream.java:326) at java.io.BufferedOutputStream.write(BufferedOutputStream.java:122) at java.io.BufferedOutputStream.write(BufferedOutputStream.java:122) at org.apache.nifi.stream.io.StreamUtils.copy(StreamUtils.java:36) at org.apache.nifi.processors.standard.ExecuteStreamCommand$2.run(ExecuteStreamCommand.java:519) at java.lang.Thread.run(Thread.java:748) 2019-09-23 12:51:54,570 ERROR [Timer-Driven Process Thread-27] o.a.n.p.standard.ExecuteStreamCommand ExecuteStreamCommand[id=a7cb604a-465d-3440-beab-b9554e1ba4bb] Transferring flow file StandardFlowFileRecord[uuid=e5a82bb6-23a9-4a16-825a-1d976cd114cc,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1569242062433-4406, container=default, section=310], offset=680454, length=-1],offset=0,name=164088,size=0] to nonzero status. Executable command /usr/bin/python3 ended in an error: 2019-09-23 12:59:56,012 INFO [main] org.apache.nifi.NiFi Launching NiFi... 2019-09-23 12:59:56,179 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Determined default nifi.properties path to be '/opt/nifi/./conf/nifi.properties' 2019-09-23 12:59:56,181 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Loaded 149 properties from /opt/nifi/./conf/nifi.properties 2019-09-23 12:59:56,186 INFO [main] org.apache.nifi.NiFi Loaded 149 properties This electronic transmission (and any attachments thereto) is intended solely for the use of the addressee(s). It may contain confidential or legally privileged information. If you are not the intended recipient of this message, you must delete it immediately and notify the sender. Any unauthorized use or disclosure of this message is strictly prohibited. Faurecia does not guarantee the integrity of this transmission and shall therefore never be liable if the message is altered or falsified nor for any virus, interception or damage to your system.
