Jim Witschey created CASSANDRA-9131:
---------------------------------------
Summary: Defining correct behavior during leap second insertion
Key: CASSANDRA-9131
URL: https://issues.apache.org/jira/browse/CASSANDRA-9131
Project: Cassandra
Issue Type: Bug
Environment: Linux ip-172-31-0-5 3.2.0-57-virtual #87-Ubuntu SMP Tue
Nov 12 21:53:49 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
Reporter: Jim Witschey
On Linux platforms, the insertion of a leap second breaks the monotonicity of
timestamps. This can make values appear to have been inserted into Cassandra in
a different order than they were. I want to know what behavior is expected and
desirable for inserts over this discontinuity.
>From a timestamp perspective, an inserted leap second looks like a repeat of
>the previous second:
{code}
$ while true ; do echo "`date +%s%N` `date -u`" ; sleep .5 ; done
1435708798171327029 Tue Jun 30 23:59:58 UTC 2015
1435708798679392477 Tue Jun 30 23:59:58 UTC 2015
1435708799187550335 Tue Jun 30 23:59:59 UTC 2015
1435708799695670453 Tue Jun 30 23:59:59 UTC 2015
1435708799203902068 Tue Jun 30 23:59:59 UTC 2015
1435708799712168566 Tue Jun 30 23:59:59 UTC 2015
1435708800220473932 Wed Jul 1 00:00:00 UTC 2015
1435708800728908190 Wed Jul 1 00:00:00 UTC 2015
1435708801237611983 Wed Jul 1 00:00:01 UTC 2015
1435708801746251996 Wed Jul 1 00:00:01 UTC 2015
{code}
Note that 23:59:59 repeats itself, and that the timestamps increase during the
first time through, then step back down to the beginning of the second and
increase again.
As a result, the timestamps on values inserted during these seconds will be out
of order. I set up a 4-node cluster running under Ubuntu 12.04.3 and synced
them to shortly before the leap second would be inserted. During the insertion
of the leap second, I ran a test with logic something like:
{code}
simple_insert = session.prepare(
'INSERT INTO test (foo, bar) VALUES (?, ?);')
for i in itertools.count():
# stop after midnight
now = datetime.utcnow()
last_midnight = now.replace(hour=0, minute=0,
second=0, microsecond=0)
seconds_since_midnight = (now - last_midnight).total_seconds()
if 5 <= seconds_since_midnight <= 15:
break
session.execute(simple_insert, [i, i])
result = session.execute("SELECT bar, WRITETIME(bar) FROM test;")
{code}
Under normal circumstances, the values and writetimes would increase together,
but when inserted over the leap second, they don't. These {{value, writetime}}
pairs are sorted by writetime:
{code}
(582, 1435708799285000)
(579, 1435708799339000)
(583, 1435708799593000)
(580, 1435708799643000)
(584, 1435708799897000)
(581, 1435708799958000)
{code}
The values were inserted in increasing order, but their writetimes are in a
different order because of the repeated second. During the first instance of
23:59:59, the values 579, 580, and 581 were inserted at the beginning, middle,
and end of the second. During the leap second, which is also 23:59:59, 582,
583, and 584 were inserted, also at the beginning, middle, and end of the
second. However, since the two seconds are the same second, they appear
interleaved with respect to timestamps, as shown above.
So, should I consider this behavior correct? If not, how should Cassandra
correctly handle the discontinuity introduced by the insertion of a leap second?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
