Github user jvwing commented on the issue:
https://github.com/apache/nifi/pull/239
@mans2singh, the error messages did not contain anything helpful that
indicated it was a memory issue, just the text
`com.amazonaws.services.kinesis.producer.DaemonException: The child process has
been shutdown and can no longer accept messages`. Memory was discussed on one
of the KPL issue tickets you found (thanks!), so I tried it out.
I'm not sure if we can do anything to handle this, but I don't know Kinesis
well. One option would be to try to restart the child process when this error
happens. But I may have actually been doing that. I stopped and started the
PutKinesisStream processor several times, which recreates the KinesisProducer
instance, and the [KinesisProducer
javadoc](https://github.com/awslabs/amazon-kinesis-producer/blob/master/java/amazon-kinesis-producer/src/main/java/com/amazonaws/services/kinesis/producer/KinesisProducer.java)
claims this spawns a new child processor. This might have actually worked,
briefly, just that the new child process died shortly afterwards because of the
same memory constraint. So even if we automated that troubleshooting step in
PutKinesisStream, the outcome for this particular issue would be the same.
Part of transparently using the KPL/KCL is that users will have to know how to
troubleshoot them directly.
There is no special trick to getting NiFi to work on EC2, their Linux VMs
are very similar to any other. I am used to EC2, and find it convenient for
testing. EC2's "micro" sized instances, which I use because I am terribly
cheap, double as a handy way to test resource constraints. Have you had
trouble with it?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
