As a general rule, partitions can certainly be much larger than 1 MB, even up
to 100 MB. 5 MB to 10 MB might be a good target size.
Originally you stated that the number of seq_types could be “unlimited”... is
that really true? Is there no practical upper limit you can establish, like
10,000 or 10 million or...? Sure, buckets are a very real option, but if the
number of seq_types was only 10,000 to 50,000, then bucketing might be
unnecessary complexity and access overhead.
-- Jack Krupansky
From: Kai Wang
Sent: Sunday, December 7, 2014 3:06 PM
To: user@cassandra.apache.org
Subject: Re: How to model data to achieve specific data locality
Thanks for the help. I wasn't clear how clustering column works. Coming from
Thrift experience, it took me a while to understand how clustering column
impacts partition storage on disk. Now I believe using seq_type as the first
clustering column solves my problem. As of partition size, I will start with
some bucket assumption. If the partition size exceeds the threshold I may need
to re-bucket using smaller bucket size.
On another thread Eric mentions the optimal partition size should be at 100 kb
~ 1 MB. I will use that as the start point to design my bucket strategy.
On Sun, Dec 7, 2014 at 10:32 AM, Jack Krupansky <j...@basetechnology.com> wrote:
It would be helpful to look at some specific examples of sequences, showing
how they grow. I suspect that the term “sequence” is being overloaded in some
subtly misleading way here.
Besides, we’ve already answered the headline question – data locality is
achieved by having a common partition key. So, we need some clarity as to what
question we are really focusing on
And, of course, we should be asking the “Cassandra Data Modeling 101”
question of what do your queries want to look like, how exactly do you want to
access your data. Only after we have a handle on how you need to read your data
can we decide how it should be stored.
My immediate question to get things back on track: When you say “The typical
read is to load a subset of sequences with the same seq_id”, what type of
“subset” are you talking about? Again, a few explicit and concise example
queries (in some concise, easy to read pseudo language or even plain English,
but not belabored with full CQL syntax.) would be very helpful. I mean,
Cassandra has no “subset” concept, nor a “load subset” command, so what are we
really talking about?
Also, I presume we are talking CQL, but some of the references seem more
Thrift/slice oriented.
-- Jack Krupansky
From: Eric Stevens
Sent: Sunday, December 7, 2014 10:12 AM
To: user@cassandra.apache.org
Subject: Re: How to model data to achieve specific data locality
> Also new seq_types can be added and old seq_types can be deleted. This
means I often need to ALTER TABLE to add and drop columns.
Kai, unless I'm misunderstanding something, I don't see why you need to alter
the table to add a new seq type. From a data model perspective, these are just
new values in a row.
If you do have columns which are specific to particular seq_types, data
modeling does become a little more challenging. In that case you may get some
advantage from using collections (especially map) to store data which applies
to only a few seq types. Or defining a schema which includes the set of all
possible columns (that's when you're getting into ALTERs when a new column
comes or goes).
> All sequences with the same seq_id tend to grow at the same rate.
Note that it is an anti pattern in Cassandra to append to the same row
indefinitely. I think you understand this because of your original question.
But please note that a sub partitioning strategy which reuses subpartitions
will result in degraded read performance after a while. You'll need to rotate
sub partitions by something that doesn't repeat in order to keep the data for a
given partition key grouped into just a few sstables. A typical pattern there
is to use some kind of time bucket (hour, day, week, etc., depending on your
write volume).
I do note that your original question was about preserving data locality -
and having a consistent locality for a given seq_id - for best offline
analytics. If you wanted to work for this, you can certainly also include a
blob value in your partitioning key, whose value is calculated to force a ring
collision with this record's sibling data. With Cassandra's default
partitioner of murmur3, that's probably pretty challenging - murmur3 isn't
designed to be cryptographically strong (it doesn't work to make it difficult
to force a collision), but it's meant to have good distribution (it may still
be computationally expensive to force a collision - I'm not that familiar with
its internal workings). In this case, ByteOrderedPartitioner would be a lot
easier to force a ring collision on, but then you need to work on a good ring
balancing strategy to distribute your data evenly over the ring.
On Sun Dec 07 2014 at 2:56:26 AM DuyHai Doan <doanduy...@gmail.com> wrote:
"Those sequences are not fixed. All sequences with the same seq_id tend to
grow at the same rate. If it's one partition per seq_id, the size will most
likely exceed the threshold quickly"
--> Then use bucketing to avoid too wide partitions
"Also new seq_types can be added and old seq_types can be deleted. This
means I often need to ALTER TABLE to add and drop columns. I am not sure if
this is a good practice from operation point of view."
--> I don't understand why altering table is necessary to add seq_types. If
"seq_types" is defined as your clustering column, you can have many of them
using the same table structure ...
On Sat, Dec 6, 2014 at 10:09 PM, Kai Wang <dep...@gmail.com> wrote:
On Sat, Dec 6, 2014 at 11:18 AM, Eric Stevens <migh...@gmail.com> wrote:
It depends on the size of your data, but if your data is reasonably
small, there should be no trouble including thousands of records on the same
partition key. So a data model using PRIMARY KEY ((seq_id), seq_type) ought to
work fine.
If the data size per partition exceeds some threshold that represents
the right tradeoff of increasing repair cost, gc pressure, threatening
unbalanced loads, and other issues that come with wide partitions, then you can
subpartition via some means in a manner consistent with your work load, with
something like PRIMARY KEY ((seq_id, subpartition), seq_type).
For example, if seq_type can be processed for a given seq_id in any
order, and you need to be able to locate specific records for a known
seq_id/seq_type pair, you can compute subpartition is computed
deterministically. Or if you only ever need to read all values for a given
seq_id, and the processing order is not important, just randomly generate a
value for subpartition at write time, as long as you can know all possible
values for subpartition.
If the values for the seq_types for a given seq_id must always be
processed in order based on seq_type, then your subpartition calculation would
need to reflect that and place adjacent seq_types in the same partition. As a
contrived example, say seq_type was an incrementing integer, your subpartition
could be seq_type / 100.
On Fri Dec 05 2014 at 7:34:38 PM Kai Wang <dep...@gmail.com> wrote:
I have a data model question. I am trying to figure out how to model
the data to achieve the best data locality for analytic purpose. Our
application processes sequences. Each sequence has a unique key in the format
of [seq_id]_[seq_type]. For any given seq_id, there are unlimited number of
seq_types. The typical read is to load a subset of sequences with the same
seq_id. Naturally I would like to have all the sequences with the same seq_id
to co-locate on the same node(s).
However I can't simply create one partition per seq_id and use seq_id
as my partition key. That's because:
1. there could be thousands or even more seq_types for each seq_id.
It's not feasible to include all the seq_types into one table.
2. each seq_id might have different sets of seq_types.
3. each application only needs to access a subset of seq_types for a
seq_id. Based on CASSANDRA-5762, select partial row loads the whole row. I
prefer only touching the data that's needed.
As per above, I think I should use one partition per
[seq_id]_[seq_type]. But how can I archive the data locality on seq_id? One
possible approach is to override IPartitioner so that I just use part of the
field (say 64 bytes) to get the token (for location) while still using the
whole field as partition key (for look up). But before heading that direction,
I would like to see if there are better options out there. Maybe any new or
upcoming features in C* 3.0?
Thanks.
Thanks, Eric.
Those sequences are not fixed. All sequences with the same seq_id tend to
grow at the same rate. If it's one partition per seq_id, the size will most
likely exceed the threshold quickly. Also new seq_types can be added and old
seq_types can be deleted. This means I often need to ALTER TABLE to add and
drop columns. I am not sure if this is a good practice from operation point of
view.
I thought about your subpartition idea. If there are only a few
applications and each one of them uses a subset of seq_types, I can easily
create one table per application since I can compute the subpartition
deterministically as you said. But in my case data scientists need to easily
write new applications using any combination of seq_types of a seq_id. So I
want the data model to be flexible enough to support applications using any
different set of seq_types without creating new tables, duplicate all the data
etc.
-Kai
