[
https://issues.apache.org/jira/browse/FLINK-9009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16413587#comment-16413587
]
Pankaj commented on FLINK-9009:
-------------------------------
CassandraSink when given parallelism=10 at enviornment level. It creates 10
different cluster for a stream.
Is there any possibility that all threads can reuse one single cluster object.
Cause I checked and found that every thread is executing Line82 of
CassandraSinkBase.java
this.cluster = builder.getCluster();
So, In Case when you have for an example 5 streams in a single Flink
enviornment with parallelism=10 and all those 5 streams are required to be
persisted in Cassandra. This would result in 50 cluster object in a single
flink run time enviornment.
I think all threads which belong to a stream should share the cluster object.
This is very important to understand and to solve "LEAK: You are creating too
many HashedWheelTimer instances. HashedWheelTimer is a shared resource that
must be reused across the JVM,so that only a few instances are created."
Please correct if i mistaken anything.
> Error| You are creating too many HashedWheelTimer instances.
> HashedWheelTimer is a shared resource that must be reused across the
> application, so that only a few instances are created.
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-9009
> URL: https://issues.apache.org/jira/browse/FLINK-9009
> Project: Flink
> Issue Type: Bug
> Environment: Pass platform: Openshift
> Reporter: Pankaj
> Priority: Major
>
> Steps to reproduce:
> 1- Flink with Kafka as a consumer -> Writing stream to Cassandra using flink
> cassandra sink.
> 2- In memory Job manager and task manager with checkpointing 5000ms.
> 3- env.setpararllelism(10)-> As kafka topic has 10 partition.
> 4- There are around 13 unique streams in a single flink run time environment
> which are reading from kafka -> processing and writing to cassandra.
> Hardware: CPU 200 milli core . It is deployed on Paas platform on one node
> Memory: 526 MB.
>
> When i start the server, It starts flink and all off sudden stops with above
> error. It also shows out of memory error.
>
> It would be nice if any body can suggest if something is wrong.
>
> Maven:
> flink-connector-cassandra_2.11: 1.3.2
> flink-streaming-java_2.11: 1.4.0
> flink-connector-kafka-0.11_2.11:1.4.0
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
