Hello Flume users, I'm getting this error when starting the agent. The source is a rabbit mq that has millions of messages, channel is file and sink is HDFS..
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "RabbitMQ Consumer #0" Exception in thread "SinkRunner-PollingRunner-DefaultSinkProcessor" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "pool-8-thread-1" java.lang.OutOfMemoryError: GC overhead limit exceeded ^CException in thread "Thread-0" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "agent-shutdown-hook" java.lang.OutOfMemoryError: GC overhead limit exceeded I have tried increasing the JAVA_OPTS min and max in flume-env.sh but that has not helped. Any help appreciated. Thank you Suresh.
