[
https://issues.apache.org/jira/browse/FLINK-26793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518077#comment-17518077
]
Etienne Chauchot commented on FLINK-26793:
------------------------------------------
[~bumblebee], in the meantime, I made a [pure-flink test
pipeline|https://github.com/echauchot/flink/blob/FLINK-26793-cassandra-perf/flink-connectors/flink-connector-cassandra/src/test/java/org/apache/flink/streaming/connectors/cassandra/example/CassandraPojoSinkStreamingExample.java]
that runs in streaming mode (so it is infinite) with a fake source that
provides Pojos every 100 ms and write them to Cassandra. I've run this pipeline
locally for 2 hours with no perf log messages. I'll try to run them for 72
hours or more on my Azure environment (hoping that I can get a Cassandra
instance there). That way, I'll try to reproduce the performance issue logs you
had. If I can't reproduce after some days of running this pipeline, I think the
good way to proceed is to take a look at your Flink restore stats hence my
question above or take a look at the Scylla connector you use.
> Flink Cassandra connector performance issue
> --------------------------------------------
>
> Key: FLINK-26793
> URL: https://issues.apache.org/jira/browse/FLINK-26793
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Cassandra
> Affects Versions: 1.14.4
> Reporter: Jay Ghiya
> Assignee: Etienne Chauchot
> Priority: Major
>
> A warning is observed during long runs of flink job stating “Insertions into
> scylla might be suffering. Expect performance problems unless this is
> resolved.”
> Upon initial analysis - “flink cassandra connector is not keeping instance of
> mapping manager that is used to convert a pojo to cassandra row. Ideally the
> mapping manager should have the same life time as cluster and session objects
> which are also created once when the driver is initialized”
> Reference:
> https://stackoverflow.com/questions/59203418/cassandra-java-driver-warning
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
