[jira] [Commented] (BEAM-4049) Improve write throughput of CassandraIO
[ https://issues.apache.org/jira/browse/BEAM-4049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16647064#comment-16647064 ] Arun sethia commented on BEAM-4049: --- I can still see the problem - https://issues.apache.org/jira/browse/BEAM-5723 > Improve write throughput of CassandraIO > --- > > Key: BEAM-4049 > URL: https://issues.apache.org/jira/browse/BEAM-4049 > Project: Beam > Issue Type: Improvement > Components: io-java-cassandra >Affects Versions: 2.4.0 >Reporter: Alexander Dejanovski >Assignee: Alexander Dejanovski >Priority: Major > Labels: performance > Fix For: 2.5.0 > > Time Spent: 8.5h > Remaining Estimate: 0h > > The CassandraIO currently uses the mapper to perform writes in a synchronous > fashion. > This implies that writes are serialized and is a very suboptimal way of > writing to Cassandra. > The IO should use the saveAsync() method instead of save() and should wait > for completion each time 100 queries are in flight, in order to avoid > overwhelming clusters. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-4049) Improve write throughput of CassandraIO
[ https://issues.apache.org/jira/browse/BEAM-4049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435072#comment-16435072 ] Alexander Dejanovski commented on BEAM-4049: PR sent : [https://github.com/apache/beam/pull/5112] I had to exclude Guava's ListenableFuture from the relocation to avoid exceptions at runtime since the DS java driver uses it in saveAsync(). > Improve write throughput of CassandraIO > --- > > Key: BEAM-4049 > URL: https://issues.apache.org/jira/browse/BEAM-4049 > Project: Beam > Issue Type: Improvement > Components: io-java-cassandra >Affects Versions: 2.4.0 >Reporter: Alexander Dejanovski >Assignee: Jean-Baptiste Onofré >Priority: Major > Labels: performance > Time Spent: 10m > Remaining Estimate: 0h > > The CassandraIO currently uses the mapper to perform writes in a synchronous > fashion. > This implies that writes are serialized and is a very suboptimal way of > writing to Cassandra. > The IO should use the saveAsync() method instead of save() and should wait > for completion each time 100 queries are in flight, in order to avoid > overwhelming clusters. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-4049) Improve write throughput of CassandraIO
[ https://issues.apache.org/jira/browse/BEAM-4049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433671#comment-16433671 ] Alexander Dejanovski commented on BEAM-4049: [~jbonofre]: I have a patch in the works so you can assign me this ticket if you want to. > Improve write throughput of CassandraIO > --- > > Key: BEAM-4049 > URL: https://issues.apache.org/jira/browse/BEAM-4049 > Project: Beam > Issue Type: Improvement > Components: io-java-cassandra >Affects Versions: 2.4.0 >Reporter: Alexander Dejanovski >Assignee: Jean-Baptiste Onofré >Priority: Major > Labels: performance > > The CassandraIO currently uses the mapper to perform writes in a synchronous > fashion. > This implies that writes are serialized and is a very suboptimal way of > writing to Cassandra. > The IO should use the saveAsync() method instead of save() and should wait > for completion each time 100 queries are in flight, in order to avoid > overwhelming clusters. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
