Hi all, I am new to flink. I tried to start a cluster over 3 servers, and the process speed was GREAT. However, after several hours' streaming, I got this error:
*java.lang.OutOfMemoryError: unable to create new native thread* From: at org.apache.flink.streaming.connectors.fs.StreamWriterBase.open(StreamWriterBase.java:120) at org.apache.flink.streaming.connectors.fs.StringWriter.open(StringWriter.java:62) It seems that when flink was writing to HDFS, flink opened too many files which run out all the native threads. I wonder is there a way for BucketingSink to solve this. Or how can I optimize my setting in general? Thanks in advance. FYI, I also posted this question to stack overflow: https://stackoverflow.com/questions/44425709/flink-unable-to-create-new-native-thread-when-use-bucketingsink
