Re: SELECT some_column vs SELECT *

2015-11-24 Thread Jon Haddad
If it's sparsely populated you'll get the same benefit from the schema 
definition.  You don't pay for fields you don't use.

> On Nov 24, 2015, at 12:17 PM, Jack Krupansky  wrote:
> 
> Are all or ost of the 1000+ columns populated for a given row? If they are 
> sparse you can replace them with a single map collection column which would 
> only occupy the entries that are populated.
> 
> -- Jack Krupansky
> 
> On Tue, Nov 24, 2015 at 11:04 AM, Jack Krupansky  > wrote:
> As always, your queries should drive your data model. Unless you really need 
> 1000+ columns for most queries, you should consider separate tables for the 
> subsets of the columns that need to be returned for a given query.
> 
> The new 3.0 Materialized View feature can be used to easily create subsets of 
> a base table, although that begs the question of whether you ever really need 
> all 1000+ columns in the same table.
> 
> -- Jack Krupansky
> 
> On Tue, Nov 24, 2015 at 10:45 AM, Kai Wang  > wrote:
> Hi all,
> 
> If I have the following table:
> CREATE TABLE t (
>   pk int,
>   ck int,
>   c1 int,
>   c2 int,
>   ...
>   PRIMARY KEY (pk, ck)
> )
> 
> There are lots of non-clustering columns (1000+). From time to time I need to 
> do a query like this:
> 
> SELECT c1 FROM t WHERE pk = abc AND ck > xyz;
> 
> How efficient is this query compared to SELECT * ...? Apparently SELECT c1 
> would save a lot of network bandwidth since only c1 needs to be transferred 
> on the wire. But I am more interested in the impact on disk IO. If I 
> understand C* storage engine correctly, one CQL row is clustered together on 
> disk. That means c1 from different rows are stored apart. In the case of 
> SELECT c1, does C* do multiple seeks to only lift c1 of each row from disk or 
> lift the whole row into memory and return c1 from there?
> 
> From comments on https://issues.apache.org/jira/browse/CASSANDRA-5762 
>  it seems C* lifts the 
> whole row as of 1.2.7. Is this still the case on 2.1.*?
> 
> Thanks.
> 
> 



Re: Many keyspaces pattern

2015-11-24 Thread Jack Krupansky
And DateTieredCompactionStrategy can be used to efficiently remove whole
sstables when the TTL expires, but this implies knowing what TTL to set in
advance.

I don't know if there are any tools to bulk delete older than a specific
age when DateTieredCompactionStrategy is used, but it might be a nice
feature.

-- Jack Krupansky

On Tue, Nov 24, 2015 at 12:53 PM, Saladi Naidu 
wrote:

> I can think of following features to solve
>
> 1. If you know the time period of after how long data should be removed
> then use TTL feature
> 2. Use Time Series to model the data and use inverted index to query the
> data by time period?
>
> Naidu Saladi
>
>
>
> On Tuesday, November 24, 2015 6:49 AM, Jack Krupansky <
> jack.krupan...@gmail.com> wrote:
>
>
> How often is sometimes - closer to 20% of the batches or 2%?
>
> How are you querying batches, both current and older ones?
>
> As always, your queries should drive your data models.
>
> If deleting a batch is very infrequent, maybe best to not do it and simply
> have logic in the app to ignore deleted batches - if your queries would
> reference them at all.
>
> What reasons would you have to delete a batch? Depending on the nature of
> the reason there may be an alternative.
>
> Make sure your cluster is adequately provisioned so that these expensive
> operations can occur in parallel to reduce their time and resources per
> node.
>
> Do all batches eventually get aged and deleted or are you expecting that
> most batches will live for many years to come? Have you planned for how you
> will grow the cluster over time?
>
> Maybe bite the bullet and use a background process to delete a batch if
> deletion is competing too heavily with query access - if they really need
> to be deleted at all.
>
> Number of keyspaces - and/or tables - should be limited to "low hundreds",
> and even then you are limited by RAM and CPU of each node. If a keyspace
> has 14 tables, then 250/14 = 20 would be a recommended upper limit for
> number of key spaces. Even if your total number of tables was under 300 or
> even 200, you would need to do a proof of concept implementation to verify
> that your specific data works well on your specific hardware.
>
>
> -- Jack Krupansky
>
> On Tue, Nov 24, 2015 at 5:05 AM, Jonathan Ballet 
> wrote:
>
> Hi,
>
> we are running an application which produces every night a batch with
> several hundreds of Gigabytes of data. Once a batch has been computed, it
> is never modified (nor updates nor deletes), we just keep producing new
> batches every day.
>
> Now, we are *sometimes* interested to remove a complete specific batch
> altogether. At the moment, we are accumulating all these data into only one
> keyspace which has a batch ID column in all our tables which is also part
> of the primary key. A sample table looks similar to this:
>
>   CREATE TABLE computation_results (
>   batch_id int,
>   id1 int,
>   id2 int,
>   value double,
>   PRIMARY KEY ((batch_id, id1), id2)
>   ) WITH CLUSTERING ORDER BY (id2 ASC);
>
> But we found out it is very difficult to remove a specific batch as we
> need to know all the IDs to delete the entries and it's both time and
> resource consuming (ie. it takes a long time and I'm not sure it's going to
> scale at all.)
>
> So, we are currently looking into having each of our batches in a keyspace
> of their own so removing a batch is merely equivalent to delete a keyspace.
> Potentially, it means we will end up having several hundreds of keyspaces
> in one cluster, although most of the time only the very last one will be
> used (we might still want to access the older ones, but that would be a
> very seldom use-case.) At the moment, the keyspace has about 14 tables and
> is probably not going to evolve much.
>
>
> Are there any counter-indications of using lot of keyspaces (300+) into
> one Cassandra cluster?
> Are there any good practices that we should follow?
> After reading the "Anti-patterns in Cassandra > Too many keyspaces or
> tables", does it mean we should plan ahead to already split our keyspace
> among several clusters?
>
> Finally, would there be any other way to achieve what we want to do?
>
> Thanks for your help!
>
>  Jonathan
>
>
>
>
>


Re: SELECT some_column vs SELECT *

2015-11-24 Thread Jack Krupansky
Are all or ost of the 1000+ columns populated for a given row? If they are
sparse you can replace them with a single map collection column which would
only occupy the entries that are populated.

-- Jack Krupansky

On Tue, Nov 24, 2015 at 11:04 AM, Jack Krupansky 
wrote:

> As always, your queries should drive your data model. Unless you really
> need 1000+ columns for most queries, you should consider separate tables
> for the subsets of the columns that need to be returned for a given query.
>
> The new 3.0 Materialized View feature can be used to easily create subsets
> of a base table, although that begs the question of whether you ever really
> need all 1000+ columns in the same table.
>
> -- Jack Krupansky
>
> On Tue, Nov 24, 2015 at 10:45 AM, Kai Wang  wrote:
>
>> Hi all,
>>
>> If I have the following table:
>> CREATE TABLE t (
>>   pk int,
>>   ck int,
>>   c1 int,
>>   c2 int,
>>   ...
>>   PRIMARY KEY (pk, ck)
>> )
>>
>> There are lots of non-clustering columns (1000+). From time to time I
>> need to do a query like this:
>>
>> SELECT c1 FROM t WHERE pk = abc AND ck > xyz;
>>
>> How efficient is this query compared to SELECT * ...? Apparently SELECT
>> c1 would save a lot of network bandwidth since only c1 needs to be
>> transferred on the wire. But I am more interested in the impact on disk IO.
>> If I understand C* storage engine correctly, one CQL row is clustered
>> together on disk. That means c1 from different rows are stored apart. In
>> the case of SELECT c1, does C* do multiple seeks to only lift c1 of each
>> row from disk or lift the whole row into memory and return c1 from there?
>>
>> From comments on https://issues.apache.org/jira/browse/CASSANDRA-5762 it
>> seems C* lifts the whole row as of 1.2.7. Is this still the case on 2.1.*?
>>
>> Thanks.
>>
>
>


Re: Many keyspaces pattern

2015-11-24 Thread Saladi Naidu
I can think of following features to solve
1. If you know the time period of after how long data should be removed then 
use TTL feature2. Use Time Series to model the data and use inverted index to 
query the data by time period? Naidu Saladi 
 


On Tuesday, November 24, 2015 6:49 AM, Jack Krupansky 
 wrote:
 

 How often is sometimes - closer to 20% of the batches or 2%?
How are you querying batches, both current and older ones?
As always, your queries should drive your data models.
If deleting a batch is very infrequent, maybe best to not do it and simply have 
logic in the app to ignore deleted batches - if your queries would reference 
them at all.
What reasons would you have to delete a batch? Depending on the nature of the 
reason there may be an alternative.
Make sure your cluster is adequately provisioned so that these expensive 
operations can occur in parallel to reduce their time and resources per node.
Do all batches eventually get aged and deleted or are you expecting that most 
batches will live for many years to come? Have you planned for how you will 
grow the cluster over time?
Maybe bite the bullet and use a background process to delete a batch if 
deletion is competing too heavily with query access - if they really need to be 
deleted at all.
Number of keyspaces - and/or tables - should be limited to "low hundreds", and 
even then you are limited by RAM and CPU of each node. If a keyspace has 14 
tables, then 250/14 = 20 would be a recommended upper limit for number of key 
spaces. Even if your total number of tables was under 300 or even 200, you 
would need to do a proof of concept implementation to verify that your specific 
data works well on your specific hardware.

-- Jack Krupansky
On Tue, Nov 24, 2015 at 5:05 AM, Jonathan Ballet  wrote:

Hi,

we are running an application which produces every night a batch with several 
hundreds of Gigabytes of data. Once a batch has been computed, it is never 
modified (nor updates nor deletes), we just keep producing new batches every 
day.

Now, we are *sometimes* interested to remove a complete specific batch 
altogether. At the moment, we are accumulating all these data into only one 
keyspace which has a batch ID column in all our tables which is also part of 
the primary key. A sample table looks similar to this:

  CREATE TABLE computation_results (
      batch_id int,
      id1 int,
      id2 int,
      value double,
      PRIMARY KEY ((batch_id, id1), id2)
  ) WITH CLUSTERING ORDER BY (id2 ASC);

But we found out it is very difficult to remove a specific batch as we need to 
know all the IDs to delete the entries and it's both time and resource 
consuming (ie. it takes a long time and I'm not sure it's going to scale at 
all.)

So, we are currently looking into having each of our batches in a keyspace of 
their own so removing a batch is merely equivalent to delete a keyspace. 
Potentially, it means we will end up having several hundreds of keyspaces in 
one cluster, although most of the time only the very last one will be used (we 
might still want to access the older ones, but that would be a very seldom 
use-case.) At the moment, the keyspace has about 14 tables and is probably not 
going to evolve much.


Are there any counter-indications of using lot of keyspaces (300+) into one 
Cassandra cluster?
Are there any good practices that we should follow?
After reading the "Anti-patterns in Cassandra > Too many keyspaces or tables", 
does it mean we should plan ahead to already split our keyspace among several 
clusters?

Finally, would there be any other way to achieve what we want to do?

Thanks for your help!

 Jonathan




  

[ANNOUNCE] YCSB 0.5.0 Release

2015-11-24 Thread Connor McCoy
On behalf of the development community, I am pleased to announce the
release of YCSB 0.5.0.

Highlights:

Added support for Kudu (getkudu.io).
Added CQL support for Cassandra 2.1+, via the cassandra2-cql binding.
Improved semantics for scans under JDBC.
Replaced numeric return codes with meaningful status messages.

Full release notes, including links to source and convenience binaries:

https://github.com/brianfrankcooper/YCSB/releases/tag/0.5.0

This release covers changes from the last 2 months.

Best,
Connor


RE: No query results while expecting results

2015-11-24 Thread Peer, Oded
Ramon,

Have you tried another driver to determine if the problem is in the Python 
driver?

You can deserialize your composite key using the following code:

  ByteBuffer t = 
ByteBufferUtil.hexToBytes("0008000e70451f6404000500");

  short periodlen = t.getShort();
  byte[] periodbuf = new byte[periodlen];
  t.get(periodbuf);
  BigInteger period = new BigInteger(periodbuf);
  System.out.println( "period " + period );
  t.get(); // null marker

  short tnt_idlen = t.getShort();
  byte[] tnt_idbuf = new byte[tnt_idlen];
  t.get(tnt_idbuf);
  BigInteger tnt_id = new BigInteger(tnt_idbuf);
  System.out.println( "tnt_id " + tnt_id.toString() );
  t.get();

The output is:
period 62013120356
tnt_id 5



From: Carlos Alonso [mailto:i...@mrcalonso.com]
Sent: Monday, November 23, 2015 9:00 PM
To: user@cassandra.apache.org
Subject: Re: No query results while expecting results

Did you tried to observe it using cassandra-cli? (the thrift client)
It shows the 'disk-layout' of the data and may help as well.

Otherwise, if you can reproduce it having a varint as the last part of the 
partition key (or at any other location), this may well be a bug.

Carlos Alonso | Software Engineer | @calonso

On 23 November 2015 at 18:48, Ramon Rockx 
> wrote:
Hello Carlos,

On Mon, Nov 23, 2015 at 3:31 PM, Carlos Alonso 
> wrote:
Well, this makes me wonder how varints are compared in java vs python because 
the problem may be there.

I'd suggest getting the token, to know which server contains the missing data. 
Go there and convert sstables to json, find the record and see what's there as 
the tnt_id. You could also use the thrift client to list it and see how it 
looks on disk and see if there's something wrong.

If the data is there and looks fine, probably there's a problem managing 
varints somewhere in the read path.

Thanks for your input.
I converted the sstables to json and found the record, it starts with:

{"key": "0008000e70451f6404000500","columns": 
[["cba56260-5c1c-11e3-bf53-402d20524153:1","{\"v\":1383400,\"s\":2052461,\"r\"...8<...

It's a composite key and I don't know how to deserialize it properly.
Maybe I can setup a test case to reproduce the problem.

Thanks,
Ramon



Re: Strategy tools for taking snapshots to load in another cluster instance

2015-11-24 Thread Anishek Agarwal
Peer,

that talks about having a similar sized cluster, I was wondering if there
is a way for moving from larger to smaller cluster. I will try a few things
as soon as i get time and update here.

On Thu, Nov 19, 2015 at 5:48 PM, Peer, Oded  wrote:

> Have you read the DataStax documentation?
>
>
> http://docs.datastax.com/en/cassandra/2.0/cassandra/operations/ops_snapshot_restore_new_cluster.html
>
>
>
>
>
> *From:* Romain Hardouin [mailto:romainh...@yahoo.fr]
> *Sent:* Wednesday, November 18, 2015 3:59 PM
> *To:* user@cassandra.apache.org
> *Subject:* Re: Strategy tools for taking snapshots to load in another
> cluster instance
>
>
>
> You can take a snapshot via nodetool then load sstables on your test
> cluster with sstableloader:
> docs.datastax.com/en/cassandra/2.1/cassandra/tools/toolsBulkloader_t.html
>
>
>
> Sent from Yahoo Mail on Android
> 
> --
>
> *From*:"Anishek Agarwal" 
> *Date*:Wed, Nov 18, 2015 at 11:24
> *Subject*:Strategy tools for taking snapshots to load in another cluster
> instance
>
> Hello
>
>
>
> We have 5 node prod cluster and 3 node test cluster. Is there a way i can
> take snapshot of a table in prod and load it test cluster. The cassandra
> versions are same.
>
>
>
> Even if there is a tool that can help with this it will be great.
>
>
>
> If not, how do people handle scenarios where data in prod is required in
> staging/test clusters for testing to make sure things are correct ? Does
> the cluster size have to be same to allow copying of relevant snapshot data
> etc?
>
>
>
>
>
> thanks
>
> anishek
>
>
>


Many keyspaces pattern

2015-11-24 Thread Jonathan Ballet

Hi,

we are running an application which produces every night a batch with 
several hundreds of Gigabytes of data. Once a batch has been computed, 
it is never modified (nor updates nor deletes), we just keep producing 
new batches every day.


Now, we are *sometimes* interested to remove a complete specific batch 
altogether. At the moment, we are accumulating all these data into only 
one keyspace which has a batch ID column in all our tables which is also 
part of the primary key. A sample table looks similar to this:


  CREATE TABLE computation_results (
  batch_id int,
  id1 int,
  id2 int,
  value double,
  PRIMARY KEY ((batch_id, id1), id2)
  ) WITH CLUSTERING ORDER BY (id2 ASC);

But we found out it is very difficult to remove a specific batch as we 
need to know all the IDs to delete the entries and it's both time and 
resource consuming (ie. it takes a long time and I'm not sure it's going 
to scale at all.)


So, we are currently looking into having each of our batches in a 
keyspace of their own so removing a batch is merely equivalent to delete 
a keyspace. Potentially, it means we will end up having several hundreds 
of keyspaces in one cluster, although most of the time only the very 
last one will be used (we might still want to access the older ones, but 
that would be a very seldom use-case.) At the moment, the keyspace has 
about 14 tables and is probably not going to evolve much.



Are there any counter-indications of using lot of keyspaces (300+) into 
one Cassandra cluster?

Are there any good practices that we should follow?
After reading the "Anti-patterns in Cassandra > Too many keyspaces or 
tables", does it mean we should plan ahead to already split our keyspace 
among several clusters?


Finally, would there be any other way to achieve what we want to do?

Thanks for your help!

 Jonathan


Re: No query results while expecting results

2015-11-24 Thread Ramon Rockx
Hello Carlos and Oded,

Thanks to you all for your input!
@Carlos, I did not try the thrift client yet.
@Oded, thank you for deserializing the key. It looks exactly what to
expect, once it's deserialized...
I think we're onto something. I reproduced and simplified the case like
this. First I created a table like this:

CREATE TABLE mytesttable (
  tnt_id varint PRIMARY KEY,
  data text
) WITH COMPACT STORAGE;

I wrote two test clients.

*Hector*
First one uses Hector as a client, which we have used often in our
application.
Here some code:

public class HectorVarIntTest {

final static String KEYSPACE = "sandbox";
final static String CLUSTERNAME = "Test Cluster";
final static String COLUMNFAMILY = "mytesttable";
final static String HOST = "localhost:9160";

static protected Keyspace getKeyspaceInstance() {
return HFactory.createKeyspace(KEYSPACE,
HFactory.getOrCreateCluster(CLUSTERNAME, HOST));
}

public void insert(final Keyspace keyspace, final Integer partitionKey,
final String value) {
final HColumn column =
HFactory.createColumn("data", value);

final Mutator mutator = HFactory.createMutator(keyspace,
IntegerSerializer.get());
mutator.addInsertion(partitionKey, COLUMNFAMILY, column);
mutator.execute();
}

public static void main(final String[] args) {
final Keyspace keyspace = getKeyspaceInstance();
final HectorVarIntTest test = new HectorVarIntTest();
test.insert(keyspace, 5, "hector test value");
}

}

Note: when running this test against the same table *without* the COMPACT
STORAGE, the test would fail with an error like this:
InvalidRequestException(why:Not
enough bytes to read value of component 0)

*Java Driver 2.1 for Apache Cassandra*
The other client is using the Datastax Java driver (CQL), to which we are
migrating.
public class CqlVarIntTest {

final static String KEYSPACE = "sandbox";
final static String CLUSTERNAME = "Test Cluster";
final static String TABLENAME = "mytesttable";
final static String HOST = "localhost";

public static void main(final String[] args) {
final Cluster cluster =
Cluster.builder().addContactPoints(HOST).build();
try {
final Session session = cluster.connect(KEYSPACE);
try {
session.execute("INSERT INTO " + TABLENAME + "(tnt_id,
data) VALUES (5, 'cql test value')");
} finally {
session.close();
}
} finally {
cluster.close();
}
}

}

I think this last test would have been the same as executing the query in
the CQL shell client, but I wanted to be sure how it was done by the Java
driver.

After running both tests this is what the table looks like:
cqlsh:sandbox> select * from mytesttable;

 tnt_id | data
+---
  5 | hector test value
  5 |cql test value

Interesting... Two records with the same *tnt_id*. That should not be
possible in this case?!

When converting the relevant sstable to json, we see this:
[
{"key": "05","columns": [["data","cql test value",1448361315125000]]},
{"key": "0005","columns": [["data","hector test
value",1448361219355000]]}
]

So the keys are different after all. Hector is serializing varints
differently then the CQL Java Driver does.
But shouldn't the Cassandra host reject the Hector key in the first place?
Or should the Cassandra host re-serialize it so it's always stored in the
same way?

And does somebody know how to workaround this? How can I query these keys
inserted by Hector with the CQL (Java) client?

Thanks again!

Ramon


Re: Many keyspaces pattern

2015-11-24 Thread Jack Krupansky
How often is sometimes - closer to 20% of the batches or 2%?

How are you querying batches, both current and older ones?

As always, your queries should drive your data models.

If deleting a batch is very infrequent, maybe best to not do it and simply
have logic in the app to ignore deleted batches - if your queries would
reference them at all.

What reasons would you have to delete a batch? Depending on the nature of
the reason there may be an alternative.

Make sure your cluster is adequately provisioned so that these expensive
operations can occur in parallel to reduce their time and resources per
node.

Do all batches eventually get aged and deleted or are you expecting that
most batches will live for many years to come? Have you planned for how you
will grow the cluster over time?

Maybe bite the bullet and use a background process to delete a batch if
deletion is competing too heavily with query access - if they really need
to be deleted at all.

Number of keyspaces - and/or tables - should be limited to "low hundreds",
and even then you are limited by RAM and CPU of each node. If a keyspace
has 14 tables, then 250/14 = 20 would be a recommended upper limit for
number of key spaces. Even if your total number of tables was under 300 or
even 200, you would need to do a proof of concept implementation to verify
that your specific data works well on your specific hardware.


-- Jack Krupansky

On Tue, Nov 24, 2015 at 5:05 AM, Jonathan Ballet  wrote:

> Hi,
>
> we are running an application which produces every night a batch with
> several hundreds of Gigabytes of data. Once a batch has been computed, it
> is never modified (nor updates nor deletes), we just keep producing new
> batches every day.
>
> Now, we are *sometimes* interested to remove a complete specific batch
> altogether. At the moment, we are accumulating all these data into only one
> keyspace which has a batch ID column in all our tables which is also part
> of the primary key. A sample table looks similar to this:
>
>   CREATE TABLE computation_results (
>   batch_id int,
>   id1 int,
>   id2 int,
>   value double,
>   PRIMARY KEY ((batch_id, id1), id2)
>   ) WITH CLUSTERING ORDER BY (id2 ASC);
>
> But we found out it is very difficult to remove a specific batch as we
> need to know all the IDs to delete the entries and it's both time and
> resource consuming (ie. it takes a long time and I'm not sure it's going to
> scale at all.)
>
> So, we are currently looking into having each of our batches in a keyspace
> of their own so removing a batch is merely equivalent to delete a keyspace.
> Potentially, it means we will end up having several hundreds of keyspaces
> in one cluster, although most of the time only the very last one will be
> used (we might still want to access the older ones, but that would be a
> very seldom use-case.) At the moment, the keyspace has about 14 tables and
> is probably not going to evolve much.
>
>
> Are there any counter-indications of using lot of keyspaces (300+) into
> one Cassandra cluster?
> Are there any good practices that we should follow?
> After reading the "Anti-patterns in Cassandra > Too many keyspaces or
> tables", does it mean we should plan ahead to already split our keyspace
> among several clusters?
>
> Finally, would there be any other way to achieve what we want to do?
>
> Thanks for your help!
>
>  Jonathan
>


SELECT some_column vs SELECT *

2015-11-24 Thread Kai Wang
Hi all,

If I have the following table:
CREATE TABLE t (
  pk int,
  ck int,
  c1 int,
  c2 int,
  ...
  PRIMARY KEY (pk, ck)
)

There are lots of non-clustering columns (1000+). From time to time I need
to do a query like this:

SELECT c1 FROM t WHERE pk = abc AND ck > xyz;

How efficient is this query compared to SELECT * ...? Apparently SELECT c1
would save a lot of network bandwidth since only c1 needs to be transferred
on the wire. But I am more interested in the impact on disk IO. If I
understand C* storage engine correctly, one CQL row is clustered together
on disk. That means c1 from different rows are stored apart. In the case of
SELECT c1, does C* do multiple seeks to only lift c1 of each row from disk
or lift the whole row into memory and return c1 from there?

>From comments on https://issues.apache.org/jira/browse/CASSANDRA-5762 it
seems C* lifts the whole row as of 1.2.7. Is this still the case on 2.1.*?

Thanks.


Re: SELECT some_column vs SELECT *

2015-11-24 Thread Jack Krupansky
As always, your queries should drive your data model. Unless you really
need 1000+ columns for most queries, you should consider separate tables
for the subsets of the columns that need to be returned for a given query.

The new 3.0 Materialized View feature can be used to easily create subsets
of a base table, although that begs the question of whether you ever really
need all 1000+ columns in the same table.

-- Jack Krupansky

On Tue, Nov 24, 2015 at 10:45 AM, Kai Wang  wrote:

> Hi all,
>
> If I have the following table:
> CREATE TABLE t (
>   pk int,
>   ck int,
>   c1 int,
>   c2 int,
>   ...
>   PRIMARY KEY (pk, ck)
> )
>
> There are lots of non-clustering columns (1000+). From time to time I need
> to do a query like this:
>
> SELECT c1 FROM t WHERE pk = abc AND ck > xyz;
>
> How efficient is this query compared to SELECT * ...? Apparently SELECT c1
> would save a lot of network bandwidth since only c1 needs to be transferred
> on the wire. But I am more interested in the impact on disk IO. If I
> understand C* storage engine correctly, one CQL row is clustered together
> on disk. That means c1 from different rows are stored apart. In the case of
> SELECT c1, does C* do multiple seeks to only lift c1 of each row from disk
> or lift the whole row into memory and return c1 from there?
>
> From comments on https://issues.apache.org/jira/browse/CASSANDRA-5762 it
> seems C* lifts the whole row as of 1.2.7. Is this still the case on 2.1.*?
>
> Thanks.
>