I am not sure, but looks it will cause the update other than insert. If it is true, the only way is request includes IF NOT EXISTS, inform the client it failed?
Thanks, Linbo 2016-06-15 10:59 GMT+08:00 Ben Slater <ben.sla...@instaclustr.com>: > Is it possible that your pub_timestamp values are colliding (which would > result in an update rather than an insert)? > > On Wed, 15 Jun 2016 at 12:55 linbo liao <llbg...@gmail.com> wrote: > >> Hi, >> >> I use Erlang driver to send data to Cassandra, do testing at local >> environment meet data lost issue. I have no idea what step is wrong. >> >> *Environment:* >> >> 1. Ubuntu 12.04 LTS x64bit >> 2. Cassandra 3.5 single instance, not a cluster, installed via the >> offical installation document, and didn't change any configuration except >> enable the authenticator and authorizer. >> 3. Cassandra binary protocol v4 >> 3. Latest Erlang driver https://github.com/matehat/cqerl >> 4. Erlang OTP 18.3 >> >> *Schema:* >> >> > DESCRIBE TABLE message.history >>> >>> CREATE TABLE message.history ( >>> appkey text, >>> pub_date text, >>> pub_timestamp timestamp, >>> apns text, >>> message blob, >>> message_id text, >>> pub_method smallint, >>> qos smallint, >>> recv_type smallint, >>> topic text, >>> PRIMARY KEY ((appkey, pub_date), pub_timestamp) >>> ) WITH CLUSTERING ORDER BY (pub_timestamp ASC) >>> AND bloom_filter_fp_chance = 0.01 >>> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'} >>> AND comment = '' >>> AND compaction = {'class': >>> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', >>> 'max_threshold': '32', 'min_threshold': '4', >>> 'unchecked_tombstone_compaction': 'true'} >>> AND compression = {'chunk_length_in_kb': '64', 'class': >>> 'org.apache.cassandra.io.compress.LZ4Compressor'} >>> AND crc_check_chance = 1.0 >>> AND dclocal_read_repair_chance = 0.1 >>> AND default_time_to_live = 0 >>> AND gc_grace_seconds = 3600 >>> AND max_index_interval = 2048 >>> AND memtable_flush_period_in_ms = 0 >>> AND min_index_interval = 128 >>> AND read_repair_chance = 0.0 >>> AND speculative_retry = '99PERCENTILE'; >>> >> >> >> *Issue:* >> >> The client send 6 insert request to Server (ttl is 1000s), check the TCP >> package every thing works fine, but data will miss random at Server >> side(select insert data was missing). >> >> >> > select * from history; >>> >>> appkey | pub_date | pub_timestamp >>> | apns | message >>> | message_id | pub_method | >>> qos | recv_type | topic >>> >>> --------------------------+------------+---------------------------------+--------------------------------------------------------------+-------------------------------------------- >>> >>> ----------------------------------+----------------------+------------+-----+-----------+----------------- >>> 52fcc04c4dc903d66d6f8f92 | 2016-06-13 | 2016-06-13 09:16:24.700000+0000 >>> | | >>> 0x68656c6c6f20746f20616c6961732066726f6d207 >>> 075626c697368325f746f5f616c696173 | 589020307122032641 | 2 >>> | 1 | 1 | alias_mqttc_sub >>> 52fcc04c4dc903d66d6f8f92 | 2016-06-13 | 2016-06-13 09:16:24.817000+0000 >>> | {"aps":{"sound":"bingbong.aiff","badge":3,"alert":"douban"}} >>> | 0x7b22612 >>> 23a2066726f6d207075626c697368327d | 11833652203486491113 | 2 >>> | 1 | 0 | t2thi >>> 52fcc04c4dc903d66d6f8f92 | 2016-06-13 | 2016-06-13 09:16:24.818000+0000 >>> | | >>> 0x66726f6d20707974686f6e | 589020307579211776 | 2 >>> | 1 | 0 | testtopic2 >>> 52fcc04c4dc903d66d6f8f92 | 2016-06-13 | 2016-06-13 09:16:24.890000+0000 >>> | {"aps":{"sound":"bingbong.aiff","badge":3,"alert":"douban"}} | >>> 0x66726f6d207075626c69736832 | 589020307814092800 | 2 >>> | 1 | 0 | testtopic2 >>> 52fcc04c4dc903d66d6f8f92 | 2016-06-13 | 2016-06-13 09:16:25.024000+0000 >>> | | >>> 0x68656c6c6f20746f20616c696173 | 589020307818287105 | 2 >>> | 1 | 1 | mytestalias1 >>> >> >> >> *TCP package inserted succeed flow:* >> >> 17:16:24.818210 IP localhost.38918 > localhost.9042: Flags [P.], seq >>> 1953472577:1953472814, ack 1963420469, win 530, options [nop,nop,TS val >>> 72982868 ecr 72940042], length 237 >>> 0x0000: 4500 0121 e5b1 4000 4006 5623 7f00 0001 E..!..@.@.V#.... >>> 0x0010: 7f00 0001 9806 2352 746f a041 7507 6b35 ......#Rto.Au.k5 >>> 0x0020: 8018 0212 ff15 0000 0101 080a 0459 a154 .............Y.T >>> 0x0030: 0458 fa0a *0400 0000 0a00 0000 e400 1005* .X.............. >>> 0x0040: 6e06 c1fc 222c 813f 6228 61c5 7364 6500 n...",.?b(a.sde. >>> 0x0050: 0105 000b 0000 0018 3532 6663 6330 3463 ........52fcc04c >>> 0x0060: 3464 6339 3033 6436 3664 3666 3866 3932 4dc903d66d6f8f92 >>> 0x0070: 0000 000a 3230 3136 2d30 362d 3133 0000 ....2016-06-13.. >>> 0x0080: 0008 0000 0155 490c 3571 0000 0002 0001 .....UI.5q...... >>> 0x0090: 0000 0002 0002 0000 0014 3131 3833 3336 ..........118336 >>> 0x00a0: 3532 3230 3334 3836 3439 3131 3133 0000 52203486491113.. >>> 0x00b0: 0014 7b22 6122 3a20 6672 6f6d 2070 7562 ..{"a":.from.pub >>> 0x00c0: 6c69 7368 327d 0000 003c 7b22 6170 7322 lish2}...<{"aps" >>> 0x00d0: 3a7b 2273 6f75 6e64 223a 2262 696e 6762 :{"sound":"bingb >>> 0x00e0: 6f6e 672e 6169 6666 222c 2262 6164 6765 ong.aiff","badge >>> 0x00f0: 223a 332c 2261 6c65 7274 223a 2264 6f75 ":3,"alert":"dou >>> 0x0100: 6261 6e22 7d7d 0000 0005 7432 7468 6900 ban"}}....t2thi. >>> 0x0110: 0000 0200 0000 0000 0400 0003 e800 0000 ................ >>> 0x0120: 64 d >>> 17:16:24.819204 IP localhost.9042 > localhost.38918: Flags [P.], seq >>> 1:14, ack 237, win 630, options [nop,nop,TS val 72982868 ecr 72982868], >>> length 13 >>> 0x0000: 4500 0041 1733 4000 4006 2582 7f00 0001 E..A.3@.@.%..... >>> 0x0010: 7f00 0001 2352 9806 7507 6b35 746f a12e ....#R..u.k5to.. >>> 0x0020: 8018 0276 fe35 0000 0101 080a 0459 a154 ...v.5.......Y.T >>> 0x0030: 0459 a154 *8400 0000 0800 0000 0400 0000* .Y.T............ >>> 0x0040: * 01 * . >>> 17:16:24.819238 IP localhost.38918 > localhost.9042: Flags [.], ack 14, >>> win 530, options [nop,nop,TS val 72982868 ecr 72982868], length 0 >>> 0x0000: 4500 0034 e5b2 4000 4006 570f 7f00 0001 E..4..@.@.W..... >>> 0x0010: 7f00 0001 9806 2352 746f a12e 7507 6b42 ......#Rto..u.kB >>> 0x0020: 8010 0212 fe28 0000 0101 080a 0459 a154 .....(.......Y.T >>> 0x0030: 0459 a154 .Y.T >>> >> >> >> >> >> *The missing TCP flow:* >>> >>> 17:16:24.820392 IP localhost.49128 > localhost.9042: Flags [P.], seq >>> 202:396, ack 14, win 530, options [nop,nop,TS val 72982868 ecr 72982840], >>> length 194 >>> 0x0000: 4500 00f6 370c 4000 4006 04f4 7f00 0001 E...7.@.@....... >>> 0x0010: 7f00 0001 bfe8 2352 83eb 4b09 bfe6 87a4 ......#R..K..... >>> 0x0020: 8018 0212 feea 0000 0101 080a 0459 a154 .............Y.T >>> 0x0030: 0459 a138 *0400 0000 0a00 0000 b900 1005* .Y.8............ >>> 0x0040: 6e06 c1fc 222c 813f 6228 61c5 7364 6500 n...",.?b(a.sde. >>> 0x0050: 0105 000b 0000 0018 3532 6663 6330 3463 ........52fcc04c >>> 0x0060: 3464 6339 3033 6436 3664 3666 3866 3932 4dc903d66d6f8f92 >>> 0x0070: 0000 000a 3230 3136 2d30 362d 3133 0000 ....2016-06-13.. >>> 0x0080: 0008 0000 0155 490c 3572 0000 0002 0001 .....UI.5r...... >>> 0x0090: 0000 0002 0002 0000 0014 3131 3833 3336 ..........118336 >>> 0x00a0: 3532 3230 3334 3836 3439 3131 3132 0000 52203486491112.. >>> 0x00b0: 0020 6672 6f6d 2070 7974 686f 6e20 7769 ..from.python.wi >>> 0x00c0: 7468 2067 6976 656e 206d 6573 7361 6765 th.given.message >>> 0x00d0: 4964 0000 0000 0000 000a 7465 7374 746f Id........testto >>> 0x00e0: 7069 6332 0000 0002 0000 0000 0004 0000 pic2............ >>> 0x00f0: 03e8 0000 0064 .....d >>> 17:16:24.834414 IP localhost.9042 > localhost.49128: Flags [P.], seq >>> 14:27, ack 396, win 579, options [nop,nop,TS val 72982872 ecr 72982868], >>> length 13 >>> 0x0000: 4500 0041 f907 4000 4006 43ad 7f00 0001 E..A..@.@.C..... >>> 0x0010: 7f00 0001 2352 bfe8 bfe6 87a4 83eb 4bcb ....#R........K. >>> 0x0020: 8018 0243 fe35 0000 0101 080a 0459 a158 ...C.5.......Y.X >>> 0x0030: 0459 a154 *8400 0000 0800 0000 0400 0000* .Y.T............ >>> 0x0040: *01* . >>> 17:16:24.834452 IP localhost.49128 > localhost.9042: Flags [.], ack 27, >>> win 530, options [nop,nop,TS val 72982872 ecr 72982872], length 0 >>> 0x0000: 4500 0034 370d 4000 4006 05b5 7f00 0001 E..47.@.@....... >>> 0x0010: 7f00 0001 bfe8 2352 83eb 4bcb bfe6 87b1 ......#R..K..... >>> 0x0020: 8010 0212 fe28 0000 0101 080a 0459 a158 .....(.......Y.X >>> 0x0030: 0459 a158 .Y.X >>> >> >> >> Check the TCP flow, the data send and ack succeed, but not sure why the >> data doesn't insert in Server side successful. Any tool or method can I >> debug the Cassandra Server, or am I missing something? >> >> Thanks, >> Linbo >> > -- > ———————— > Ben Slater > Chief Product Officer > Instaclustr: Cassandra + Spark - Managed | Consulting | Support > +61 437 929 798 >
