Re: batchstatement
What is the primary key for the user_by_ext table? I'd assume it's ext_id,
which would imply your update doesn't make sense as you can't change the
primary key for a row - which would be the problem you're seeing.
On Sat., 14 Jul. 2018, 06:14 Randy Lynn, wrote:
> TL/DR:
> - only 1 out of 14 statements in a batch do not mutate the partition..
> - no error is logged in the application layer, or Cassandra system.log or
> Cassandra debug.log
> - using C#, and Datastax latest driver
> - cluster is a 1-node, dev setup.
> - datastax driver configured with LOCAL_QUORUM at the session, and
> statement level.
> - using preparedstatements.. 1,000% sure there's no typo.. (but I've been
> wrong before)
>
>
> I have about 14 statements that get batched up together. They're updating
> at most 2, maybe 3 denormalized tables.. all the same user object, just
> different lookup keys.
>
> To help visualize, the tables look a little like these.. abbreviated..
> User table..
> CREATE TABLE user (u_id uuid, act_id uuid, ext_id text, dt_created
> timeuuid, dt_mod timeuuid, is_group Boolean, first_name text)
>
> Users By Account (or plan)
> CREATE TABLE user_by_act (act_id uuid, u_id uuid, first_name text)
>
> User By external identifier
> CREATE TABLE user_by_ext (ext_id text, u_id uuid, act_id uuid, first_name
> text)
>
> I create a batch that updates all the tables.. various updates are broken
> out into separate statements, so for example, there's a statement that
> updates the external ID in the 'user' table.
>
> UPDATE user_by_ext SET ext_id = :ext_id WHERE u_id = :u_id
>
> This particular batch has 14 statements total, across all 3 tables. They
> are only updating at most 3 partitions.. a single partition may have 4 or
> more statements to update various parts of the partition. e.g. first name
> and last name are a single statement added to the batch.
>
> Here's the problem... of those 14 statements.. across the 3 partitions...
> ONE and ONLY ONE update doesn't work.. Absolutely every other discreet
> update in the whole batch works.
>
> List boundStatements = new
> List();
>
> // *
> // user table
>
> boundStatements.Add(SessionManager.UserInsertStatement.Bind(new { u_id=
> user.UserId, act_id = user.ActId, dt_created = nowId, dt_mod = nowId,
> is_everyone = user.IsEveryone, is_group = user.IsGroup }));
>
> if (!string.IsNullOrWhiteSpace(user.ExtId))
> //
> // this statement gets added to the list.. it is part of the batch
> // but it NEVER updates the actual field in the databse.
> // I have moved it around.. up, down... the only thing that works
> // is if I call execute on the first binding above, and then add the rest
> // of these as a separate batch.
>
> boundStatements.Add(SessionManager.UserUpdateExtIdStatement.Bind(new { u_id
> = user.UserId, ext_id = user.ExtId, dt_mod = nowId }));
> //
>
>
> if (!string.IsNullOrWhiteSpace(user.Email))
>
> boundStatements.Add(SessionManager.UserUpdateEmailStatement.Bind(new { u_id
> = user.UserId, email = user.Email, dt_mod = nowId }));
> BoundStatement userProfile =
> CreateUserProfileBoundStatement(nowId, user);
> if (userProfile != null)
> boundStatements.Add(userProfile);
> // *
> // user_by_act table
> CreateUserAccountInsertBoundStatements(boundStatements, user,
> nowId);
> // *
> // user_by_ext table
> if (!string.IsNullOrWhiteSpace(user.ExtId))
> {
>
> boundStatements.Add(SessionManager.UserExtInsertStatement.Bind(new { ext_id
> = user.ExtId, act_id = user.ActId, dt_created = nowId, dt_mod = nowId,
> is_group = user.IsGroup, u_id = user.UserId }));
> BoundStatement userByExtProfile =
> CreateUserByExtProfileBoundStatement(nowId, user);
> if (userByExtProfile != null)
> boundStatements.Add(userByExtProfile);
> if (!string.IsNullOrWhiteSpace(user.Email))
>
> boundStatements.Add(SessionManager.UserExtUpdateEmailStatement.Bind(new {
> ext_id = user.ExtId, email = user.Email, dt_mod = nowId }));
> }
>
>
>
> --
> Randy Lynn
> rl...@getavail.com
>
> office:
> 859.963.1616 <+1-859-963-1616> ext 202
> 163 East Main Street - Lexington, KY 40507 - USA
>
>
batchstatement
TL/DR:
- only 1 out of 14 statements in a batch do not mutate the partition..
- no error is logged in the application layer, or Cassandra system.log or
Cassandra debug.log
- using C#, and Datastax latest driver
- cluster is a 1-node, dev setup.
- datastax driver configured with LOCAL_QUORUM at the session, and
statement level.
- using preparedstatements.. 1,000% sure there's no typo.. (but I've been
wrong before)
I have about 14 statements that get batched up together. They're updating
at most 2, maybe 3 denormalized tables.. all the same user object, just
different lookup keys.
To help visualize, the tables look a little like these.. abbreviated..
User table..
CREATE TABLE user (u_id uuid, act_id uuid, ext_id text, dt_created
timeuuid, dt_mod timeuuid, is_group Boolean, first_name text)
Users By Account (or plan)
CREATE TABLE user_by_act (act_id uuid, u_id uuid, first_name text)
User By external identifier
CREATE TABLE user_by_ext (ext_id text, u_id uuid, act_id uuid, first_name
text)
I create a batch that updates all the tables.. various updates are broken
out into separate statements, so for example, there's a statement that
updates the external ID in the 'user' table.
UPDATE user_by_ext SET ext_id = :ext_id WHERE u_id = :u_id
This particular batch has 14 statements total, across all 3 tables. They
are only updating at most 3 partitions.. a single partition may have 4 or
more statements to update various parts of the partition. e.g. first name
and last name are a single statement added to the batch.
Here's the problem... of those 14 statements.. across the 3 partitions...
ONE and ONLY ONE update doesn't work.. Absolutely every other discreet
update in the whole batch works.
List boundStatements = new
List();
// *
// user table
boundStatements.Add(SessionManager.UserInsertStatement.Bind(new
{ u_id= user.UserId, act_id = user.ActId, dt_created = nowId, dt_mod =
nowId, is_everyone = user.IsEveryone, is_group = user.IsGroup }));
if (!string.IsNullOrWhiteSpace(user.ExtId))
//
// this statement gets added to the list.. it is part of the batch
// but it NEVER updates the actual field in the databse.
// I have moved it around.. up, down... the only thing that works
// is if I call execute on the first binding above, and then add the rest
// of these as a separate batch.
boundStatements.Add(SessionManager.UserUpdateExtIdStatement.Bind(new { u_id
= user.UserId, ext_id = user.ExtId, dt_mod = nowId }));
//
if (!string.IsNullOrWhiteSpace(user.Email))
boundStatements.Add(SessionManager.UserUpdateEmailStatement.Bind(new { u_id
= user.UserId, email = user.Email, dt_mod = nowId }));
BoundStatement userProfile =
CreateUserProfileBoundStatement(nowId, user);
if (userProfile != null)
boundStatements.Add(userProfile);
// *
// user_by_act table
CreateUserAccountInsertBoundStatements(boundStatements, user,
nowId);
// *
// user_by_ext table
if (!string.IsNullOrWhiteSpace(user.ExtId))
{
boundStatements.Add(SessionManager.UserExtInsertStatement.Bind(new { ext_id
= user.ExtId, act_id = user.ActId, dt_created = nowId, dt_mod = nowId,
is_group = user.IsGroup, u_id = user.UserId }));
BoundStatement userByExtProfile =
CreateUserByExtProfileBoundStatement(nowId, user);
if (userByExtProfile != null)
boundStatements.Add(userByExtProfile);
if (!string.IsNullOrWhiteSpace(user.Email))
boundStatements.Add(SessionManager.UserExtUpdateEmailStatement.Bind(new {
ext_id = user.ExtId, email = user.Email, dt_mod = nowId }));
}
--
Randy Lynn
rl...@getavail.com
office:
859.963.1616 <+1-859-963-1616> ext 202
163 East Main Street - Lexington, KY 40507 - USA
Re: datastax java driver Batch vs BatchStatement
Hi, Query builder's Batch simply sends a QUERY message through the wire where the query string is a CQL batch statement <http://docs.datastax.com/en/cql/3.3/cql/cql_reference/batch_r.html>: "BEGIN BATCH ... APPLY BATCH". BatchStatement actually sends a BATCH message <https://github.com/apache/cassandra/blob/trunk/doc/native_protocol_v4.spec#L379> through the wire, and indeed is only available from protocol V2 onwards. Both are valid ways of executing a batch and are semantically equivalent; one big advantage of BatchStatement vs Batch is that you can group prepared statements together and execute them as a batch. However neither Batch nor BatchStatements will split big batches into smaller ones AFAIK. Thanks Alexandre On Fri, Mar 25, 2016 at 4:44 AM Jimmy Lin <y2klyf+w...@gmail.com> wrote: > Hi all, > What is the difference between datastax driver Batch and BatchStatement? > > In particular, BatchStatment call out that it needs native protocol of > version 2 or above. > What is the advantage using native protocol 2.0 for batch execution? > > Will any of these two api smart enough to split a big batch into multiple > smaller one ? > (to avoid batch_size_warn_threshold_in_kb or > batch_size_failed_threshold_in_kb > ) > > Thanks > > Batch > > https://docs.datastax.com/en/drivers/java/2.1/com/datastax/driver/core/querybuilder/Batch.html > > BatchStatement > > https://docs.datastax.com/en/drivers/java/2.1/com/datastax/driver/core/BatchStatement.html > -- Alexandre Dutra Driver & Tools Engineer @ DataStax
datastax java driver Batch vs BatchStatement
Hi all, What is the difference between datastax driver Batch and BatchStatement? In particular, BatchStatment call out that it needs native protocol of version 2 or above. What is the advantage using native protocol 2.0 for batch execution? Will any of these two api smart enough to split a big batch into multiple smaller one ? (to avoid batch_size_warn_threshold_in_kb or batch_size_failed_threshold_in_kb ) Thanks Batch https://docs.datastax.com/en/drivers/java/2.1/com/datastax/driver/core/querybuilder/Batch.html BatchStatement https://docs.datastax.com/en/drivers/java/2.1/com/datastax/driver/core/BatchStatement.html
