Re: Warning for large batch sizes with a small number of statements

[Erick Ramirez]

You can increase it if you're sure that it fits your use case. For an
explanation of why batch size vs number of statements, see the discussion
in CASSANDRA-6487. Cheers!

On Mon, Nov 13, 2017 at 6:31 PM, Tim Moore  wrote:

> Hi,
>
> I'm trying to understand some of the details of the
> batch_size_warn_threshold_in_kb/batch_size_fail_threshold_in_kb settings.
>
> Specifically, why are the thresholds measured in kb rather than the number
> of partitions affected?
>
> We have run into the limit in a situation where there is a batch with
> writes to two tables (because we wanted to ensure atomicity of the writes
> in the case of a failure). In some situations, the data inserted into one
> of these tables can be large enough to push the total batch size over the
> limit.
>
> In this specific case, we were able to rewrite things so that it could be
> split into separate statement executions with the application handling
> retry on failure so atomicity is not needed. It left us wondering, however,
> whether executing this as a batch was really problematic, or if the warning
> from Cassandra was spurious in this case.
>
> Is it really the total data size that matters, or just the number of
> affected partitions?
>
> Thanks,
> Tim
>
> --
> Tim Moore
> *Senior Engineer, Lagom, Lightbend, Inc.*
> tim.mo...@lightbend.com
> +61 420 981 589 <+61%20420%20981%20589>
> Skype: timothy.m.moore
>
> 
>

Warning for large batch sizes with a small number of statements

[Tim Moore]

Hi,

I'm trying to understand some of the details of the
batch_size_warn_threshold_in_kb/batch_size_fail_threshold_in_kb settings.

Specifically, why are the thresholds measured in kb rather than the number
of partitions affected?

We have run into the limit in a situation where there is a batch with
writes to two tables (because we wanted to ensure atomicity of the writes
in the case of a failure). In some situations, the data inserted into one
of these tables can be large enough to push the total batch size over the
limit.

In this specific case, we were able to rewrite things so that it could be
split into separate statement executions with the application handling
retry on failure so atomicity is not needed. It left us wondering, however,
whether executing this as a batch was really problematic, or if the warning
from Cassandra was spurious in this case.

Is it really the total data size that matters, or just the number of
affected partitions?

Thanks,
Tim

-- 
Tim Moore
*Senior Engineer, Lagom, Lightbend, Inc.*
tim.mo...@lightbend.com
+61 420 981 589
Skype: timothy.m.moore