Re: write time for nulls is not consistent

[Nitan Kainth]

Thank you Jeff. Please share once, the Patch is available for us to apply.

> On Jul 27, 2017, at 2:12 PM, Jeff Jirsa  wrote:
> 
> 
> 
> On 2017-07-20 08:46 (-0700), "Jeff Jirsa" wrote: 
>> 
>> 
>> On 2017-07-20 08:17 (-0700), Nitan Kainth  wrote: 
>>> Jeff,
>>> 
>>> It is really strange, look at below log, I inserted your data and then few 
>>> additional; finally, the issue is reproduced:
> 
>> This is a bug, cqlsh is displaying the timestamp from partition 't' for 
>> partition 'b'. 
>> 
>> I've created https://issues.apache.org/jira/browse/CASSANDRA-13711 based on 
>> your report. I havent yet verified if this is a cqlsh bug or a storage 
>> engine bug.
>> 
> 
> It turns out this was in the CQL layer, impacted both writetime() and ttl() 
> for 3.0/3.11/trunk. The good news is that because it was in the CQL layer, it 
> didn't impact the underlying storage engine with it's confused 
> timestamps/ttls, but would have returned invalid timestamps/ttls to clients. 
> 
> It's now patch-available (I wrote up a fix this morning), and awaiting review 
> - it'll be in the next release.
> 
> Thanks again for reporting it - this type of report (on the dev/user list or 
> via JIRA) is really helpful to the community.
> 
> - Jeff
> 
> -
> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: user-h...@cassandra.apache.org
> 


-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

Re: write time for nulls is not consistent

[Jeff Jirsa]

On 2017-07-20 08:46 (-0700), "Jeff Jirsa" wrote: 
> 
> 
> On 2017-07-20 08:17 (-0700), Nitan Kainth  wrote: 
> > Jeff,
> > 
> > It is really strange, look at below log, I inserted your data and then few 
> > additional; finally, the issue is reproduced:

> This is a bug, cqlsh is displaying the timestamp from partition 't' for 
> partition 'b'. 
> 
> I've created https://issues.apache.org/jira/browse/CASSANDRA-13711 based on 
> your report. I havent yet verified if this is a cqlsh bug or a storage engine 
> bug.
> 

It turns out this was in the CQL layer, impacted both writetime() and ttl() for 
3.0/3.11/trunk. The good news is that because it was in the CQL layer, it 
didn't impact the underlying storage engine with it's confused timestamps/ttls, 
but would have returned invalid timestamps/ttls to clients. 

It's now patch-available (I wrote up a fix this morning), and awaiting review - 
it'll be in the next release.

Thanks again for reporting it - this type of report (on the dev/user list or 
via JIRA) is really helpful to the community.

- Jeff

-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

Re: write time for nulls is not consistent

[Jeff Jirsa]

On 2017-07-20 08:17 (-0700), Nitan Kainth  wrote: 
> Jeff,
> 
> It is really strange, look at below log, I inserted your data and then few 
> additional; finally, the issue is reproduced:
> 
..
> (6 rows)
> cqlsh> insert into test.t(a) values('b');
> cqlsh> select a,b, writetime (b) from test.t;
> 
>  a | b| writetime(b)
> ---+--+--
>  z | null | null
>  a | null | null
>  e | null | null
>  r | null | null
>  w | null | null
>  t |x | 1500563698113119
>  b | null | 1500563698113119
> 
> (7 rows)
> 


Repro'd on my side too.

cqlsh> insert into test.t(a,b) values('t','x');
cqlsh> insert into test.t(a) values('b');
cqlsh> select a,b, writetime (b) from test.t;

 a | b| writetime(b)
---+--+--
 z | null | null
 e | null | null
 r | null | null
 w | null | null
 t |x | 1500565131354883
 b | null | 1500565131354883

Here's what the data on disk looks like (starting with row 'w'):

  {
"partition" : {
  "key" : [ "w" ],
  "position" : 69
},
"rows" : [
  {
"type" : "row",
"position" : 92,
"liveness_info" : { "tstamp" : "2017-07-20T04:41:59.005382Z" },
"cells" : [ ]
  }
]
  },
  {
"partition" : {
  "key" : [ "t" ],
  "position" : 93
},
"rows" : [
  {
"type" : "row",
"position" : 120,
"liveness_info" : { "tstamp" : "2017-07-20T15:38:51.354883Z" },
"cells" : [
  { "name" : "b", "value" : "x" }
]
  }
]
  },
  {
"partition" : {
  "key" : [ "b" ],
  "position" : 121
},
"rows" : [
  {
"type" : "row",
"position" : 146,
"liveness_info" : { "tstamp" : "2017-07-20T15:39:03.631297Z" },
"cells" : [ ]
  }
]
  }

This is a bug, cqlsh is displaying the timestamp from partition 't' for 
partition 'b'. 

I've created https://issues.apache.org/jira/browse/CASSANDRA-13711 based on 
your report. I havent yet verified if this is a cqlsh bug or a storage engine 
bug.


-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

Re: write time for nulls is not consistent

[Nitan Kainth]

Jeff,

It is really strange, look at below log, I inserted your data and then few 
additional; finally, the issue is reproduced:

[cqlsh 5.0.1 | Cassandra 3.0.10.1443 | DSE 5.0.4 | CQL spec 3.4.0 | Native 
protocol v4]
Use HELP for help.
cqlsh> CREATE KEYSPACE test WITH replication = {'class':'SimpleStrategy', 
'replication_factor': 1};
cqlsh> CREATE TABLE test.t ( a text primary key, b text );
cqlsh> 
cqlsh> 
cqlsh> 
cqlsh> insert into test.t(a) values('z');
cqlsh> insert into test.t(a) values('w');
cqlsh> insert into test.t(a) values('e');
cqlsh> insert into test.t(a) values('r');
cqlsh> insert into test.t(a) values('t');
cqlsh> 
cqlsh> select a,b, writetime (b) from test.t;

 a | b| writetime(b)
---+--+--
 z | null | null
 e | null | null
 r | null | null
 w | null | null
 t | null | null

(5 rows)
cqlsh> 
cqlsh> insert into test.t(a,b) values('t','x');
cqlsh> select a,b, writetime (b) from test.t;

 a | b| writetime(b)
---+--+--
 z | null | null
 e | null | null
 r | null | null
 w | null | null
 t |x | 1500563698113119

(5 rows)
cqlsh> insert into test.t(a) values('a');
cqlsh> select a,b, writetime (b) from test.t;

 a | b| writetime(b)
---+--+--
 z | null | null
 a | null | null
 e | null | null
 r | null | null
 w | null | null
 t |x | 1500563698113119

(6 rows)
cqlsh> insert into test.t(a) values('b');
cqlsh> select a,b, writetime (b) from test.t;

 a | b| writetime(b)
---+--+--
 z | null | null
 a | null | null
 e | null | null
 r | null | null
 w | null | null
 t |x | 1500563698113119
 b | null | 1500563698113119

(7 rows)

> On Jul 19, 2017, at 11:43 PM, Jeff Jirsa  wrote:
> 
> cqlsh> insert into test.t(a) values('z');
> cqlsh> insert into test.t(a) values('w');
> cqlsh> insert into test.t(a) values('e');
> cqlsh> insert into test.t(a) values('r');
> cqlsh> insert into test.t(a) values('t');
> cqlsh> select a,b, writetime (b) from test.t;

Re: write time for nulls is not consistent

[Jeff Jirsa]

This doesn't reproduce with the steps you're providing. There's clearly some 
pre-existing data in the table, and some sort of operation on that table in the 
past that isn't evident from your repro steps. 

(oss-ccm) MacBook-Pro:~ jjirsa$ ccm create test -n 1 -s -v 3.0.10
Current cluster is now: test
(oss-ccm) MacBook-Pro:~ jjirsa$ ccm node1 cqlsh
Connected to test at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 3.0.10 | CQL spec 3.4.0 | Native protocol v4]
Use HELP for help.
cqlsh> CREATE KEYSPACE test WITH replication = {'class':'SimpleStrategy', 
'replication_factor': 1};
cqlsh> CREATE TABLE test.t ( a text primary key, b text );
cqlsh> insert into test.t(a) values('z');
cqlsh> insert into test.t(a) values('w');
cqlsh> insert into test.t(a) values('e');
cqlsh> insert into test.t(a) values('r');
cqlsh> insert into test.t(a) values('t');
cqlsh> select a,b, writetime (b) from test.t;

 a | b| writetime(b)
---+--+--
 z | null | null
 e | null | null
 r | null | null
 w | null | null
 t | null | null

(5 rows)
cqlsh>

Are you able to post the json for that table somewhere (sstabledump the 
-Data.db files on disk ) ? 

- Jeff

On 2017-07-19 08:33 (-0700), Nitan Kainth  wrote: 
> Same output Kurt,
> 
> [cqlsh 5.0.1 | Cassandra 3.0.10.1443 | DSE 5.0.4 | CQL spec 3.4.0 | Native 
> protocol v4]
> Use HELP for help.
> cqlsh> select a, writetime(b) from test.t;
> 
>  a | writetime(b)
> ---+--
>  z | null
>  a | null
>  c | null
>  e | null
>  r | null
>  d | 1500400745074499
>  w | 1500400745074499
>  v | 1500400745074499
>  t | 1500400745074499
>  x | 1500400626266371
> 
> 
> > On Jul 18, 2017, at 4:41 PM, kurt greaves  wrote:
> > 
> > select a, writetime(b) from test.t
> 
> 

-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

Re: write time for nulls is not consistent

[Nitan Kainth]

Same output Kurt,

[cqlsh 5.0.1 | Cassandra 3.0.10.1443 | DSE 5.0.4 | CQL spec 3.4.0 | Native 
protocol v4]
Use HELP for help.
cqlsh> select a, writetime(b) from test.t;

 a | writetime(b)
---+--
 z | null
 a | null
 c | null
 e | null
 r | null
 d | 1500400745074499
 w | 1500400745074499
 v | 1500400745074499
 t | 1500400745074499
 x | 1500400626266371


> On Jul 18, 2017, at 4:41 PM, kurt greaves  wrote:
> 
> select a, writetime(b) from test.t

Re: write time for nulls is not consistent

[Subroto Barua]

Works fine either case on 3.0.12 

Subroto 

> On Jul 18, 2017, at 2:41 PM, kurt greaves  wrote:
> 
> can you try select a, writetime(b) from test.t
> I heard of an issue recently where cqlsh reports null incorrectly if you 
> query a column twice, wondering if it extends to this case with writetime.


-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

Re: write time for nulls is not consistent

[kurt greaves]

can you try select a, writetime(b) from test.t
I heard of an issue recently where cqlsh reports null incorrectly if you
query a column twice, wondering if it extends to this case with writetime.

Re: write time for nulls is not consistent

[Nitan Kainth]

insert into test.t (a ) values ('v’);

> On Jul 18, 2017, at 2:28 PM, Jonathan Haddad  wrote:
> 
> Since you didn't show all the queries you used, I made an assumption that 
> you've inserted other data.  Where is the query that inserted a="v"?  I don't 
> know how to answer the question if you're not actually showing how to 
> reproduce the problem.
> 
> 
> On Tue, Jul 18, 2017 at 12:24 PM Nitan Kainth  > wrote:
> Jonathan,
> 
> Please notice last rows with partition key values (w,v and t). they were 
> inserted same way and has write time values
> 
>> On Jul 18, 2017, at 2:22 PM, Jonathan Haddad > > wrote:
>> 
>> This looks like expected behavior to me.  You aren't inserting a value for 
>> b.  Since there's no value, there's also no writetime.
>> 
>> On Tue, Jul 18, 2017 at 12:15 PM Nitan Kainth > > wrote:
>> Hi,
>> 
>> We see that null columns have writetime(column) populated for few columns 
>> and shows null for few other. Is it any bug or something else?
>> 
>> 
>> CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy', 
>> 'us-east': '2'}  AND durable_writes = true;
>> 
>> CREATE TABLE test.t (
>> a text PRIMARY KEY,
>> b text
>> ) WITH bloom_filter_fp_chance = 0.01
>> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
>> AND comment = ''
>> AND compaction = {'class': 
>> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
>> 'max_threshold': '32', 'min_threshold': '4'}
>> AND compression = {'chunk_length_in_kb': '64', 'class': 
>> 'org.apache.cassandra.io.compress.LZ4Compressor'}
>> AND crc_check_chance = 1.0
>> AND dclocal_read_repair_chance = 0.1
>> AND default_time_to_live = 0
>> AND gc_grace_seconds = 864000
>> AND max_index_interval = 2048
>> AND memtable_flush_period_in_ms = 0
>> AND min_index_interval = 128
>> AND read_repair_chance = 0.0
>> AND speculative_retry = '99PERCENTILE’;
>> 
>> insert into test.t (a ) values ('z’);
>> insert into test.t (a ) values ('w’);
>> insert into test.t (a ) values ('e’);
>> insert into test.t (a ) values ('r’);
>> insert into test.t (a ) values ('t’);
>> 
>>  select a,b, writetime (b) from test.t ;
>> 
>>  a | b| writetime(b)
>> ---+--+--
>>  z | null | null
>>  a | null | null
>>  c | null | null
>>  e | null | null
>>  r | null | null
>>  d | null | 1500400745074499
>>  w | null | 1500400745074499
>>  v | null | 1500400745074499
>>  t | null | 1500400745074499
>>  x |y | 1500400626266371
> 

Re: write time for nulls is not consistent

[Jonathan Haddad]

Since you didn't show all the queries you used, I made an assumption that
you've inserted other data.  Where is the query that inserted a="v"?  I
don't know how to answer the question if you're not actually showing how to
reproduce the problem.


On Tue, Jul 18, 2017 at 12:24 PM Nitan Kainth  wrote:

> Jonathan,
>
> Please notice last rows with partition key values (w,v and t). they were
> inserted same way and has write time values
>
> On Jul 18, 2017, at 2:22 PM, Jonathan Haddad  wrote:
>
> This looks like expected behavior to me.  You aren't inserting a value for
> b.  Since there's no value, there's also no writetime.
>
> On Tue, Jul 18, 2017 at 12:15 PM Nitan Kainth  wrote:
>
>> Hi,
>>
>> We see that null columns have writetime(column) populated for few columns
>> and shows null for few other. Is it any bug or something else?
>>
>>
>> CREATE KEYSPACE test WITH replication = {'class':
>> 'NetworkTopologyStrategy', 'us-east': '2'}  AND durable_writes = true;
>>
>> CREATE TABLE test.t (
>> a text PRIMARY KEY,
>> b text
>> ) WITH bloom_filter_fp_chance = 0.01
>> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
>> AND comment = ''
>> AND compaction = {'class':
>> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
>> 'max_threshold': '32', 'min_threshold': '4'}
>> AND compression = {'chunk_length_in_kb': '64', 'class':
>> 'org.apache.cassandra.io.compress.LZ4Compressor'}
>> AND crc_check_chance = 1.0
>> AND dclocal_read_repair_chance = 0.1
>> AND default_time_to_live = 0
>> AND gc_grace_seconds = 864000
>> AND max_index_interval = 2048
>> AND memtable_flush_period_in_ms = 0
>> AND min_index_interval = 128
>> AND read_repair_chance = 0.0
>> AND speculative_retry = '99PERCENTILE’;
>>
>> insert into test.t (a ) values ('z’);
>> insert into test.t (a ) values ('w’);
>> insert into test.t (a ) values ('e’);
>> insert into test.t (a ) values ('r’);
>> insert into test.t (a ) values ('t’);
>>
>>  select a,b, writetime (b) from test.t ;
>>
>>  *a* | *b*| *writetime(b)*
>> ---+--+--
>>  *z* | *null* | *null*
>>  *a* | *null* | *null*
>>  *c* | *null* | *null*
>>  *e* | *null* | *null*
>>  *r* | *null* | *null*
>>  *d* | *null* | *1500400745074499*
>>  *w* | *null* | *1500400745074499*
>>  *v* | *null* | *1500400745074499*
>>  *t* | *null* | *1500400745074499*
>>  *x* |*y* | *1500400626266371*
>>
>
>

Re: write time for nulls is not consistent

[Nitan Kainth]

Jonathan,

Please notice last rows with partition key values (w,v and t). they were 
inserted same way and has write time values

> On Jul 18, 2017, at 2:22 PM, Jonathan Haddad  wrote:
> 
> This looks like expected behavior to me.  You aren't inserting a value for b. 
>  Since there's no value, there's also no writetime.
> 
> On Tue, Jul 18, 2017 at 12:15 PM Nitan Kainth  > wrote:
> Hi,
> 
> We see that null columns have writetime(column) populated for few columns and 
> shows null for few other. Is it any bug or something else?
> 
> 
> CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy', 
> 'us-east': '2'}  AND durable_writes = true;
> 
> CREATE TABLE test.t (
> a text PRIMARY KEY,
> b text
> ) WITH bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND comment = ''
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class': 
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE’;
> 
> insert into test.t (a ) values ('z’);
> insert into test.t (a ) values ('w’);
> insert into test.t (a ) values ('e’);
> insert into test.t (a ) values ('r’);
> insert into test.t (a ) values ('t’);
> 
>  select a,b, writetime (b) from test.t ;
> 
>  a | b| writetime(b)
> ---+--+--
>  z | null | null
>  a | null | null
>  c | null | null
>  e | null | null
>  r | null | null
>  d | null | 1500400745074499
>  w | null | 1500400745074499
>  v | null | 1500400745074499
>  t | null | 1500400745074499
>  x |y | 1500400626266371

Re: write time for nulls is not consistent

[Jonathan Haddad]

This looks like expected behavior to me.  You aren't inserting a value for
b.  Since there's no value, there's also no writetime.

On Tue, Jul 18, 2017 at 12:15 PM Nitan Kainth  wrote:

> Hi,
>
> We see that null columns have writetime(column) populated for few columns
> and shows null for few other. Is it any bug or something else?
>
>
> CREATE KEYSPACE test WITH replication = {'class':
> 'NetworkTopologyStrategy', 'us-east': '2'}  AND durable_writes = true;
>
> CREATE TABLE test.t (
> a text PRIMARY KEY,
> b text
> ) WITH bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND comment = ''
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE’;
>
> insert into test.t (a ) values ('z’);
> insert into test.t (a ) values ('w’);
> insert into test.t (a ) values ('e’);
> insert into test.t (a ) values ('r’);
> insert into test.t (a ) values ('t’);
>
>  select a,b, writetime (b) from test.t ;
>
>  *a* | *b*| *writetime(b)*
> ---+--+--
>  *z* | *null* | *null*
>  *a* | *null* | *null*
>  *c* | *null* | *null*
>  *e* | *null* | *null*
>  *r* | *null* | *null*
>  *d* | *null* | *1500400745074499*
>  *w* | *null* | *1500400745074499*
>  *v* | *null* | *1500400745074499*
>  *t* | *null* | *1500400745074499*
>  *x* |*y* | *1500400626266371*
>