Re: Trouble with using group commitlog_sync

2024-04-24 Thread Nathan Marz
have run into the bottleneck of the driver's IO thread. Try > increase the driver's connections-per-server limit to 2 or 3 if you've only > got 1 server in the cluster. Or alternatively, run two client processes in > parallel. > > > On 24/04/2024 07:19, Nathan Marz wrote: > > Tr

Re: Trouble with using group commitlog_sync

2024-04-24 Thread Nathan Marz
mplicit limits on the > number of in-flight requests, which default to 1024 requests per connection > and 1 connection per server. See > https://docs.datastax.com/en/developer/java-driver/4.17/manual/core/pooling/ > > > On 23/04/2024 23:18, Nathan Marz wrote: > > It's using the a

Re: Trouble with using group commitlog_sync

2024-04-23 Thread Nathan Marz
< user@cassandra.apache.org> wrote: > Have you checked the thread CPU utilisation of the client side? You likely > will need more than one thread to do insertion in a loop to achieve tens of > thousands of inserts per second. > > > On 23/04/2024 21:55, Nathan Marz wrote: > &

Re: Trouble with using group commitlog_sync

2024-04-23 Thread Nathan Marz
itlog at shorter intervals and allow more concurrent writes. > > > On 23/04/2024 20:43, Nathan Marz wrote: > > Thanks. I raised concurrent_writes to 128 and > set commitlog_sync_group_window to 20ms. This causes a single execute of a > BatchStatement containing

Re: Trouble with using group commitlog_sync

2024-04-23 Thread Nathan Marz
rease the value of > concurrent_writes, consider at least double or quadruple it from the > default. You'll need even higher write concurrency for longer > commitlog_sync_group_window. > > On 23/04/2024 19:26, Nathan Marz wrote: > > "batch" mode works fine. I'm havin

Re: Trouble with using group commitlog_sync

2024-04-23 Thread Nathan Marz
e the writer threads will be > unable to do extra work while waiting. You may need to increase > concurrent_writes for the same reason* > > If you want to use batch mode, at least ensure commitlog_sync_batch_window > is reasonably short. The default is 2 millisecond. > > >

Trouble with using group commitlog_sync

2024-04-23 Thread Nathan Marz
I'm doing some benchmarking of Cassandra on a single m6gd.large instance. It works fine with periodic or batch commitlog_sync options, but I'm having tons of issues when I change it to "group". I have "commitlog_sync_group_window" set to 1000ms. My client is doing writes like this (pseudocode):