Re: Should we do some releases?

[J. D. Jordan]

Lots of people thinking alike. This was just being discussed on #cassandra-dev 
https://wilderness.apache.org/channels/?f=cassandra-dev/2017-09-19#1505841128

There were a couple tickets that are almost done that people wanted to get in 
before we cut a patch release.

-Jeremiah

> On Sep 19, 2017, at 8:07 PM, Nate McCall  wrote:
> 
> It's been about 3 mos since our last set.
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
> 

Should we do some releases?

[Nate McCall]

It's been about 3 mos since our last set.

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Commitlog without header

[Dikang Gu]

Hello,

In our production cluster, we had multiple times that after a *unclean*
shutdown, cassandra sever can not start due to commit log exceptions:

2017-09-17_06:06:32.49830 ERROR 06:06:32 [main]: Exiting due to error while
processing commit log during initialization.
2017-09-17_06:06:32.49831
org.apache.cassandra.db.commitlog.CommitLogReplayer$CommitLogReplayException:
Could not read commit log descriptor in file
/data/cassandra/commitlog/CommitLog-5-1503088780367.log
2017-09-17_06:06:32.49831 at
org.apache.cassandra.db.commitlog.CommitLogReplayer.handleReplayError(CommitLogReplayer.java:634)
[apache-cassandra-2.2.5+git20170612.e1857fa.jar:2.2.5+git20170612.e1857fa]
2017-09-17_06:06:32.49831 at
org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:303)
[apache-cassandra-2.2.5+git20170612.e1857fa.jar:2.2.5+git20170612.e1857fa]
2017-09-17_06:06:32.49831 at
org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:147)
[apache-cassandra-2.2.5+git20170612.e1857fa.jar:2.2.5+git20170612.e1857fa]
2017-09-17_06:06:32.49832 at
org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:189)
[apache-cassandra-2.2.5+git20170612.e1857fa.jar:2.2.5+git20170612.e1857fa]
2017-09-17_06:06:32.49832 at
org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:169)
[apache-cassandra-2.2.5+git20170612.e1857fa.jar:2.2.5+git20170612.e1857fa]
2017-09-17_06:06:32.49832 at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:302)
[apache-cassandra-2.2.5+git20170612.e1857fa.jar:2.2.5+git20170612.e1857fa]
2017-09-17_06:06:32.49832 at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:544)
[apache-cassandra-2.2.5+git20170612.e1857fa.jar:2.2.5+git20170612.e1857fa]
2017-09-17_06:06:32.49832 at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:607)
[apache-cassandra-2.2.5+git20170612.e1857fa.jar:2.2.5+git20170612.e1857fa]

I add some logging to the CommitLogDescriptor.readHeader(), and find the
header is empty in the failure case. By empty, I mean all the fields in the
header are 0:

2017-09-19_22:43:02.22112 INFO  22:43:02 [main]: Dikang: crc: 0, checkcrc:
2077607535
2017-09-19_22:43:02.22130 INFO  22:43:02 [main]: Dikang: version: 0, id: 0,
parametersLength: 0

As a result, it did not pass the crc check, and failed the commit log
replay.

My question is: is it a known issue that some race condition can cause
empty header in commit log? If so, it should be safe just skip last commit
log with empty header, right?

As you can see, we are using Cassandra 2.2.5.

Thanks
Dikang.

Re: Compact Storage and SuperColumn Tables in 4.0/trunk

[Aleksey Yeshchenko]

4.0 should also fail startup (very early) if it still sees any non-migrated 
tables, probably.

—
AY

On 19 September 2017 at 18:35:11, J. D. Jordan (jeremiah.jor...@gmail.com) 
wrote:

Thanks for the clarification. +1 for adding a "DROP COMPACT STORAGE" option in 
3.x and then not allowing it to be specified in 4.0.  


On Sep 19, 2017, at 1:27 PM, Alex P  wrote:  

>> If we provide a way to drop the flag, but still access the data, I think 
>> that is fine and perfectly reasonable. If the proposal here is that users 
>> who have data in COMPACT STORAGE tables have no way to upgrade to 4.0 and 
>> still access that data without exporting it to a brand new table, then I am 
>> against it. Can you clarify which thing is being proposed? It is not clear 
>> to me.  
>  
>  
> A bit of details on how compact storage and all thrift tables are 
> implemented:  
>  
> When a table is created through thrift or with COMPACT STORAGE flag, it has a 
> `value` column, which is invisible when doing any CQL queries and only seen 
> through Thrift.  
> With SuperColumn families, internally (on the storage level) have a partition 
> key, clustering and a value column that has a type of `map<>`. Thrift exposes 
> it as a “normal” super column family. CQL, however, does not expose this 
> `map` column. Instead, it translates the key of the map into the second 
> clustering and a map value as a regular column.  
>  
> All of this requires quite some special-casing everywhere across the CQL 
> layer, in order to hide/show and translate the columns depending on whether 
> the table is dense, super and so on.  
>  
>  
>  
> For more details you can take a look at 8099 or 12373.  
>  
> In short: dropping a COMPACT STORAGE flag means that your tables will be 
> accessible and their internal representation (e.g. hidden value column) will 
> be exposed as if it was a normal column. No data will be lost, no data will 
> be inaccessible. You can take a look at the details of CASSANDRA-10857 if you 
> want more details.  
>  
> As regards SuperColumn families, my proposal is to have a 100% support in 
> 3.0/3.11 (LWTs, counters, all sorts of queries, exactly like they were 
> accessible through CQL in 2.2).  
>  
> There will be a clear upgrade path, but I suggest that the DROP COMPACT 
> STORAGE has to be in 3.x only.  
>  
> 4.x will still make the same data available, but expose the whole internal 
> CQL structure, together with a usually “hidden" compact value column, without 
> any legacy special-casing.  
>  
>  
>  
>  
>  
>  
>  
>> On 19 Sep 2017, at 17:23, Jeremiah D Jordan  
>> wrote:  
>>  
>> I think that all the work to support Compact Storage tables from CQL seems 
>> like wasted effort if we are going to tell people “just kidding, you have to 
>> migrate all your data”. I do not think supporting “COMPACT STORAGE” as a 
>> table option matters one way or the other. But I do think being able to read 
>> the data that was in a table created that way is something we need to have a 
>> path forward for.  
>>  
>>> since thrift is not supported on trunk/4.0, it makes it much less appealing 
>>> or even necessary  
>>  
>> I think that the fact thrift is not supported on trunk/4.0 makes accessing 
>> said data from CQL *MORE* necessary and appealing.  
>>  
>>> possibility drop a Compact Storage flag and expose them as “normal" tables, 
>>> there was an idea of removing the Compact Tables from 4.x altogether.  
>>  
>> If we provide a way to drop the flag, but still access the data, I think 
>> that is fine and perfectly reasonable. If the proposal here is that users 
>> who have data in COMPACT STORAGE tables have no way to upgrade to 4.0 and 
>> still access that data without exporting it to a brand new table, then I am 
>> against it. Can you clarify which thing is being proposed? It is not clear 
>> to me.  
>>  
>> -Jeremiah  
>>  
>>  
>>> On Sep 19, 2017, at 7:10 AM, Oleksandr Petrov >> > wrote:  
>>>  
>>> As you may know, SuperColumn Tables did not work in 3.x the way they worked 
>>> in 2.x. In order to provide everyone with a reasonable upgrade path, we've 
>>> been working on CASSANDRA-12373[1], that brings in support for SuperColumn 
>>> tables as close to 2.x as possible. The patch is planned to land 
>>> cassandra-3.0 and cassandra-3.11 branches only, since the patch for trunk 
>>> will require even more work and, since thrift is not supported on 
>>> trunk/4.0, it makes it much less appealing or even necessary. The idea 
>>> behind the support for SuperColumns was always only to allow people to 
>>> smoothly migrate off them in 3.0/3.11 world, not to have them as a primary 
>>> feature.  
>>>  
>>> SuperColumns are not the only type of Compact Table, there are more. After 
>>> CASSANDRA-8099[2], Compact Tables are special-cased and have special schema 
>>> layout with some columns hidden 

Re: Compact Storage and SuperColumn Tables in 4.0/trunk

[J. D. Jordan]

Thanks for the clarification. +1 for adding a "DROP COMPACT STORAGE" option in 
3.x and then not allowing it to be specified in 4.0.


On Sep 19, 2017, at 1:27 PM, Alex P  wrote:

>> If we provide a way to drop the flag, but still access the data, I think 
>> that is fine and perfectly reasonable.  If the proposal here is that users 
>> who have data in COMPACT STORAGE tables have no way to upgrade to 4.0 and 
>> still access that data without exporting it to a brand new table, then I am 
>> against it.  Can you clarify which thing is being proposed?  It is not clear 
>> to me.
> 
> 
> A bit of details on how compact storage and all thrift tables are implemented:
> 
> When a table is created through thrift or with COMPACT STORAGE flag, it has a 
> `value` column, which is invisible when doing any CQL queries and only seen 
> through Thrift. 
> With SuperColumn families, internally (on the storage level) have a partition 
> key, clustering and a value column that has a type of `map<>`. Thrift exposes 
> it as a “normal” super column family. CQL, however, does not expose this 
> `map` column. Instead, it translates the key of the map into the second 
> clustering and a map value as a regular column.
> 
> All of this requires quite some special-casing everywhere across the CQL 
> layer, in order to hide/show and translate the columns depending on whether 
> the table is dense, super and so on.
> 
> 
> 
> For more details you can take a look at 8099 or 12373.
> 
> In short: dropping a COMPACT STORAGE flag means that your tables will be 
> accessible and their internal representation (e.g. hidden value column) will 
> be exposed as if it was a normal column. No data will be lost, no data will 
> be inaccessible. You can take a look at the details of CASSANDRA-10857 if you 
> want more details.
> 
> As regards SuperColumn families, my proposal is to have a 100% support in 
> 3.0/3.11 (LWTs, counters, all sorts of queries, exactly like they were 
> accessible through CQL in 2.2).
> 
> There will be a clear upgrade path, but I suggest that the DROP COMPACT 
> STORAGE has to be in 3.x only. 
> 
> 4.x will still make the same data available, but expose the whole internal 
> CQL structure, together with a usually “hidden" compact value column, without 
> any legacy special-casing. 
> 
> 
> 
> 
> 
> 
> 
>> On 19 Sep 2017, at 17:23, Jeremiah D Jordan  
>> wrote:
>> 
>> I think that all the work to support Compact Storage tables from CQL seems 
>> like wasted effort if we are going to tell people “just kidding, you have to 
>> migrate all your data”.  I do not think supporting “COMPACT STORAGE” as a 
>> table option matters one way or the other.  But I do think being able to 
>> read the data that was in a table created that way is something we need to 
>> have a path forward for.
>> 
>>> since thrift is not supported on trunk/4.0, it makes it much less appealing 
>>> or even necessary
>> 
>> I think that the fact thrift is not supported on trunk/4.0 makes accessing 
>> said data from CQL *MORE* necessary and appealing.
>> 
>>> possibility drop a Compact Storage flag and expose them as “normal" tables, 
>>> there was an idea of removing the Compact Tables from 4.x altogether. 
>> 
>> If we provide a way to drop the flag, but still access the data, I think 
>> that is fine and perfectly reasonable.  If the proposal here is that users 
>> who have data in COMPACT STORAGE tables have no way to upgrade to 4.0 and 
>> still access that data without exporting it to a brand new table, then I am 
>> against it.  Can you clarify which thing is being proposed?  It is not clear 
>> to me.
>> 
>> -Jeremiah
>> 
>> 
>>> On Sep 19, 2017, at 7:10 AM, Oleksandr Petrov >> > wrote:
>>> 
>>> As you may know, SuperColumn Tables did not work in 3.x the way they worked 
>>> in 2.x. In order to provide everyone with a reasonable upgrade path, we've 
>>> been working on CASSANDRA-12373[1], that brings in support for SuperColumn 
>>> tables as close to 2.x as possible. The patch is planned to land 
>>> cassandra-3.0 and cassandra-3.11 branches only, since the patch for trunk 
>>> will require even more work and, since thrift is not supported on 
>>> trunk/4.0, it makes it much less appealing or even necessary. The idea 
>>> behind the support for SuperColumns was always only to allow people to 
>>> smoothly migrate off them in 3.0/3.11 world, not to have them as a primary 
>>> feature.
>>> 
>>> SuperColumns are not the only type of Compact Table, there are more. After 
>>> CASSANDRA-8099[2], Compact Tables are special-cased and have special schema 
>>> layout with some columns hidden from CQL, that allows them to be used from 
>>> Thrift. But, except for the fact they’re accessible from Thrift, there are 
>>> no advantages to use them with the new storage. In order to allow people to 
>>> “expose” the internal structure of the 

Re: Compact Storage and SuperColumn Tables in 4.0/trunk

[Alex P]

> If we provide a way to drop the flag, but still access the data, I think that 
> is fine and perfectly reasonable.  If the proposal here is that users who 
> have data in COMPACT STORAGE tables have no way to upgrade to 4.0 and still 
> access that data without exporting it to a brand new table, then I am against 
> it.  Can you clarify which thing is being proposed?  It is not clear to me.


A bit of details on how compact storage and all thrift tables are implemented:

When a table is created through thrift or with COMPACT STORAGE flag, it has a 
`value` column, which is invisible when doing any CQL queries and only seen 
through Thrift. 
With SuperColumn families, internally (on the storage level) have a partition 
key, clustering and a value column that has a type of `map<>`. Thrift exposes 
it as a “normal” super column family. CQL, however, does not expose this `map` 
column. Instead, it translates the key of the map into the second clustering 
and a map value as a regular column.

All of this requires quite some special-casing everywhere across the CQL layer, 
in order to hide/show and translate the columns depending on whether the table 
is dense, super and so on.



For more details you can take a look at 8099 or 12373.

In short: dropping a COMPACT STORAGE flag means that your tables will be 
accessible and their internal representation (e.g. hidden value column) will be 
exposed as if it was a normal column. No data will be lost, no data will be 
inaccessible. You can take a look at the details of CASSANDRA-10857 if you want 
more details.

As regards SuperColumn families, my proposal is to have a 100% support in 
3.0/3.11 (LWTs, counters, all sorts of queries, exactly like they were 
accessible through CQL in 2.2).

There will be a clear upgrade path, but I suggest that the DROP COMPACT STORAGE 
has to be in 3.x only. 

4.x will still make the same data available, but expose the whole internal CQL 
structure, together with a usually “hidden" compact value column, without any 
legacy special-casing. 







> On 19 Sep 2017, at 17:23, Jeremiah D Jordan  wrote:
> 
> I think that all the work to support Compact Storage tables from CQL seems 
> like wasted effort if we are going to tell people “just kidding, you have to 
> migrate all your data”.  I do not think supporting “COMPACT STORAGE” as a 
> table option matters one way or the other.  But I do think being able to read 
> the data that was in a table created that way is something we need to have a 
> path forward for.
> 
>> since thrift is not supported on trunk/4.0, it makes it much less appealing 
>> or even necessary
> 
> I think that the fact thrift is not supported on trunk/4.0 makes accessing 
> said data from CQL *MORE* necessary and appealing.
> 
>> possibility drop a Compact Storage flag and expose them as “normal" tables, 
>> there was an idea of removing the Compact Tables from 4.x altogether. 
> 
> If we provide a way to drop the flag, but still access the data, I think that 
> is fine and perfectly reasonable.  If the proposal here is that users who 
> have data in COMPACT STORAGE tables have no way to upgrade to 4.0 and still 
> access that data without exporting it to a brand new table, then I am against 
> it.  Can you clarify which thing is being proposed?  It is not clear to me.
> 
> -Jeremiah
> 
> 
>> On Sep 19, 2017, at 7:10 AM, Oleksandr Petrov > > wrote:
>> 
>> As you may know, SuperColumn Tables did not work in 3.x the way they worked 
>> in 2.x. In order to provide everyone with a reasonable upgrade path, we've 
>> been working on CASSANDRA-12373[1], that brings in support for SuperColumn 
>> tables as close to 2.x as possible. The patch is planned to land 
>> cassandra-3.0 and cassandra-3.11 branches only, since the patch for trunk 
>> will require even more work and, since thrift is not supported on trunk/4.0, 
>> it makes it much less appealing or even necessary. The idea behind the 
>> support for SuperColumns was always only to allow people to smoothly migrate 
>> off them in 3.0/3.11 world, not to have them as a primary feature.
>> 
>> SuperColumns are not the only type of Compact Table, there are more. After 
>> CASSANDRA-8099[2], Compact Tables are special-cased and have special schema 
>> layout with some columns hidden from CQL, that allows them to be used from 
>> Thrift. But, except for the fact they’re accessible from Thrift, there are 
>> no advantages to use them with the new storage. In order to allow people to 
>> “expose” the internal structure of the compact tables to make them fully 
>> accessible in CQL, CASSANDRA-10857[3] was created.
>> 
>> In the light of the fact that 4.0 will not have reasonable SuperColumn 
>> support (due to related complexity and amount of special-cases required to 
>> support it in 4.0) and a possibility drop a Compact Storage flag and expose 
>> them as “normal" tables, there 

Re: question on assigning JIRA ticket

[Jeff Jirsa]

What's your JIRA username? I'll add you as a contributor.

(Goes for anyone - please email me directly and not spam the list)



On Tue, Sep 19, 2017 at 10:03 AM, Tyagi, Preetika 
wrote:

> Thank you, Jeff. It was really helpful!
>
> Also, do I need to request an access in order to be able to assign an
> issue to myself? I cannot find an option to do that when I'm logged in.
>
> Preetika
>
> -Original Message-
> From: Jeff Jirsa [mailto:jji...@gmail.com]
> Sent: Tuesday, September 19, 2017 9:50 AM
> To: Cassandra DEV 
> Subject: Re: question on assigning JIRA ticket
>
> If it's created by someone else but not assigned, you can assign it to
> yourself and begin work
>
> If it's created by someone else and assigned to someone else, you should
> post on the ticket and ask if they mind if you work on it instead.
> Sometimes people assign and then never work on it, and they won't mind if
> you take it. Sometimes they'll have started but hit a road block, and may
> be able to give you some code to start with. Just ask before you change the
> assignee.
>
> - Jeff
>
> On Tue, Sep 19, 2017 at 9:34 AM, Tyagi, Preetika  >
> wrote:
>
> > Hi all,
> >
> > I'm trying to figure out different ways in which one can contribute to
> > Cassandra. I know one can create a ticket and assign it to himself to
> > work on it. However, is it also allowed to assign and work on an
> > already existing ticket created by someone else?
> >
> > Thanks,
> > Preetika
> >
> >
>

RE: question on assigning JIRA ticket

[Tyagi, Preetika]

Thank you, Jeff. It was really helpful!

Also, do I need to request an access in order to be able to assign an issue to 
myself? I cannot find an option to do that when I'm logged in.

Preetika

-Original Message-
From: Jeff Jirsa [mailto:jji...@gmail.com] 
Sent: Tuesday, September 19, 2017 9:50 AM
To: Cassandra DEV 
Subject: Re: question on assigning JIRA ticket

If it's created by someone else but not assigned, you can assign it to yourself 
and begin work

If it's created by someone else and assigned to someone else, you should post 
on the ticket and ask if they mind if you work on it instead.
Sometimes people assign and then never work on it, and they won't mind if you 
take it. Sometimes they'll have started but hit a road block, and may be able 
to give you some code to start with. Just ask before you change the assignee.

- Jeff

On Tue, Sep 19, 2017 at 9:34 AM, Tyagi, Preetika 
wrote:

> Hi all,
>
> I'm trying to figure out different ways in which one can contribute to 
> Cassandra. I know one can create a ticket and assign it to himself to 
> work on it. However, is it also allowed to assign and work on an 
> already existing ticket created by someone else?
>
> Thanks,
> Preetika
>
>

Re: question on assigning JIRA ticket

[Jeff Jirsa]

If it's created by someone else but not assigned, you can assign it to
yourself and begin work

If it's created by someone else and assigned to someone else, you should
post on the ticket and ask if they mind if you work on it instead.
Sometimes people assign and then never work on it, and they won't mind if
you take it. Sometimes they'll have started but hit a road block, and may
be able to give you some code to start with. Just ask before you change the
assignee.

- Jeff

On Tue, Sep 19, 2017 at 9:34 AM, Tyagi, Preetika 
wrote:

> Hi all,
>
> I'm trying to figure out different ways in which one can contribute to
> Cassandra. I know one can create a ticket and assign it to himself to work
> on it. However, is it also allowed to assign and work on an already
> existing ticket created by someone else?
>
> Thanks,
> Preetika
>
>

question on assigning JIRA ticket

[Tyagi, Preetika]

Hi all,

I'm trying to figure out different ways in which one can contribute to 
Cassandra. I know one can create a ticket and assign it to himself to work on 
it. However, is it also allowed to assign and work on an already existing 
ticket created by someone else?

Thanks,
Preetika

Re: Compact Storage and SuperColumn Tables in 4.0/trunk

[Jeremiah D Jordan]

I think that all the work to support Compact Storage tables from CQL seems like 
wasted effort if we are going to tell people “just kidding, you have to migrate 
all your data”.  I do not think supporting “COMPACT STORAGE” as a table option 
matters one way or the other.  But I do think being able to read the data that 
was in a table created that way is something we need to have a path forward for.

> since thrift is not supported on trunk/4.0, it makes it much less appealing 
> or even necessary

I think that the fact thrift is not supported on trunk/4.0 makes accessing said 
data from CQL *MORE* necessary and appealing.

> possibility drop a Compact Storage flag and expose them as “normal" tables, 
> there was an idea of removing the Compact Tables from 4.x altogether. 

If we provide a way to drop the flag, but still access the data, I think that 
is fine and perfectly reasonable.  If the proposal here is that users who have 
data in COMPACT STORAGE tables have no way to upgrade to 4.0 and still access 
that data without exporting it to a brand new table, then I am against it.  Can 
you clarify which thing is being proposed?  It is not clear to me.

-Jeremiah


> On Sep 19, 2017, at 7:10 AM, Oleksandr Petrov  
> wrote:
> 
> As you may know, SuperColumn Tables did not work in 3.x the way they worked 
> in 2.x. In order to provide everyone with a reasonable upgrade path, we've 
> been working on CASSANDRA-12373[1], that brings in support for SuperColumn 
> tables as close to 2.x as possible. The patch is planned to land 
> cassandra-3.0 and cassandra-3.11 branches only, since the patch for trunk 
> will require even more work and, since thrift is not supported on trunk/4.0, 
> it makes it much less appealing or even necessary. The idea behind the 
> support for SuperColumns was always only to allow people to smoothly migrate 
> off them in 3.0/3.11 world, not to have them as a primary feature.
> 
> SuperColumns are not the only type of Compact Table, there are more. After 
> CASSANDRA-8099[2], Compact Tables are special-cased and have special schema 
> layout with some columns hidden from CQL, that allows them to be used from 
> Thrift. But, except for the fact they’re accessible from Thrift, there are no 
> advantages to use them with the new storage. In order to allow people to 
> “expose” the internal structure of the compact tables to make them fully 
> accessible in CQL, CASSANDRA-10857[3] was created.
> 
> In the light of the fact that 4.0 will not have reasonable SuperColumn 
> support (due to related complexity and amount of special-cases required to 
> support it in 4.0) and a possibility drop a Compact Storage flag and expose 
> them as “normal" tables, there was an idea of removing the Compact Tables 
> from 4.x altogether. 
> 
> 
> Leaving Compact Storage in 3.x only will make the table metadata a bit 
> lighter and allow us to remove some special cases required for their support. 
> Doing it during the major release, provided with a reasonable upgrade path 
> (same functionality from both Thrift and CQL for all compact tables, 
> including Super Column ones) through 3.x/3.11, sounds like the best option 
> that we have right now.
> 
> It’d be good if you could voice your support of this idea (or raise possible 
> concerns, if there are any).
> 
> 
> There will be additional discussion and a proposal on how to allow “online” 
> COMPACT STORAGE flag drop in CASSANDRA-10857 later this (or the following 
> week).
> 
> Best Regards, 
> Alex
> 
> [1] https://issues.apache.org/jira/browse/CASSANDRA-12373 
> 
> [2] https://issues.apache.org/jira/browse/CASSANDRA-8099 
> 
> [3] https://issues.apache.org/jira/browse/CASSANDRA-10857 
> 
> 


-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Compact Storage and SuperColumn Tables in 4.0/trunk

[Oleksandr Petrov]

As you may know, SuperColumn Tables did not work in 3.x the way they worked in 
2.x. In order to provide everyone with a reasonable upgrade path, we've been 
working on CASSANDRA-12373[1], that brings in support for SuperColumn tables as 
close to 2.x as possible. The patch is planned to land cassandra-3.0 and 
cassandra-3.11 branches only, since the patch for trunk will require even more 
work and, since thrift is not supported on trunk/4.0, it makes it much less 
appealing or even necessary. The idea behind the support for SuperColumns was 
always only to allow people to smoothly migrate off them in 3.0/3.11 world, not 
to have them as a primary feature.

SuperColumns are not the only type of Compact Table, there are more. After 
CASSANDRA-8099[2], Compact Tables are special-cased and have special schema 
layout with some columns hidden from CQL, that allows them to be used from 
Thrift. But, except for the fact they’re accessible from Thrift, there are no 
advantages to use them with the new storage. In order to allow people to 
“expose” the internal structure of the compact tables to make them fully 
accessible in CQL, CASSANDRA-10857[3] was created.

In the light of the fact that 4.0 will not have reasonable SuperColumn support 
(due to related complexity and amount of special-cases required to support it 
in 4.0) and a possibility drop a Compact Storage flag and expose them as 
“normal" tables, there was an idea of removing the Compact Tables from 4.x 
altogether. 


Leaving Compact Storage in 3.x only will make the table metadata a bit lighter 
and allow us to remove some special cases required for their support. Doing it 
during the major release, provided with a reasonable upgrade path (same 
functionality from both Thrift and CQL for all compact tables, including Super 
Column ones) through 3.x/3.11, sounds like the best option that we have right 
now.

It’d be good if you could voice your support of this idea (or raise possible 
concerns, if there are any).


There will be additional discussion and a proposal on how to allow “online” 
COMPACT STORAGE flag drop in CASSANDRA-10857 later this (or the following week).

Best Regards, 
Alex
 
[1] https://issues.apache.org/jira/browse/CASSANDRA-12373 

[2] https://issues.apache.org/jira/browse/CASSANDRA-8099 

[3] https://issues.apache.org/jira/browse/CASSANDRA-10857