[
https://issues.apache.org/jira/browse/CASSANDRA-11950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15367172#comment-15367172
]
Stefania commented on CASSANDRA-11950:
--------------------------------------
At the moment this test is consistently failing for an entirely different
reason, sample
[here|http://cassci.datastax.com/job/trunk_large_dtest/14/testReport/junit/consistency_test/TestAvailability/test_network_topology_strategy_each_quorum/].
Despite the timeout, this is not related to the recent Netty upgrade
(CASSANDRA-12032), rather this was introduced by CASSANDRA-11971. We cannot
recycle the buffer in SP.sendMessagesToNonlocalDC, since its backing array will
be used asynchronously by the message.
Patch for 3.9 and trunk:
||3.9||trunk||
|[patch|https://github.com/stef1927/cassandra/commits/11950-3.9]|[patch|https://github.com/stef1927/cassandra/commits/11950]|
|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11950-3.9-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11950-testall/]|
|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11950-3.9-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11950-dtest/]|
[~tjake] could you review? We may also want to put it in 3.8 if we are
re-tagging.
--
As for the original issue, this looks like the race that was fixed by
CASSANDRA-12083, cc [~beobal]. Relevant log messages:
{code}
DEBUG [MigrationStage:1] 2016-06-03 14:25:27,380 Schema.java:465 - Adding
org.apache.cassandra.config.CFMetaData@5fa5b473[cfId=03b14ad0-2997-11e6-b8c7-01c3aea11be7,ksName=mytestks,cfName=users,flags=[],params=TableParams{comment=,
read_repair_chance=0.0, dclocal_read_repair_chance=0.1,
bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000,
default_time_to_live=0, memtable_flush_period_in_ms=0, min_index_interval=128,
max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' :
'ALL', 'rows_per_partition' : 'NONE'},
compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,
options={max_threshold=32, min_threshold=4}},
compression=org.apache.cassandra.schema.CompressionParams@2c86b999,
extensions={}},comparator=comparator(org.apache.cassandra.db.marshal.UTF8Type),partitionColumns=[[age
firstname lastname] |
[value]],partitionKeyColumns=[ColumnDefinition{name=userid,
type=org.apache.cassandra.db.marshal.Int32Type, kind=PARTITION_KEY,
position=0}],clusteringColumns=[ColumnDefinition{name=column1,
type=org.apache.cassandra.db.marshal.UTF8Type, kind=CLUSTERING,
position=0}],keyValidator=org.apache.cassandra.db.marshal.Int32Type,columnMetadata=[ColumnDefinition{name=column1,
type=org.apache.cassandra.db.marshal.UTF8Type, kind=CLUSTERING, position=0},
ColumnDefinition{name=firstname, type=org.apache.cassandra.db.marshal.UTF8Type,
kind=STATIC, position=-1}, ColumnDefinition{name=value,
type=org.apache.cassandra.db.marshal.BytesType, kind=REGULAR, position=-1},
ColumnDefinition{name=userid, type=org.apache.cassandra.db.marshal.Int32Type,
kind=PARTITION_KEY, position=0}, ColumnDefinition{name=lastname,
type=org.apache.cassandra.db.marshal.UTF8Type, kind=STATIC, position=-1},
ColumnDefinition{name=age, type=org.apache.cassandra.db.marshal.Int32Type,
kind=STATIC, position=-1}],droppedColumns={},triggers=[],indexes=[]] to cfIdMap
ERROR [SharedPool-Worker-1] 2016-06-03 14:25:27,460 Keyspace.java:504 -
Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7
(mytestks.users)
{code}
--
The test should pass consistently provided the problem above is fixed, but
there are still several warnings because the schema hasn't fully propagated
yet, I wonder if we should add a pause to the test after creating the tables or
if there is something else that we can do to remove the warnings.
> dtest failure in
> consistency_test.TestAvailability.test_network_topology_strategy_each_quorum
> ---------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-11950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11950
> Project: Cassandra
> Issue Type: Bug
> Reporter: Sean McCarthy
> Assignee: Stefania
> Labels: dtest
> Fix For: 3.x
>
> Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log,
> node3.log, node3_debug.log, node4.log, node4_debug.log, node5.log,
> node5_debug.log, node6.log, node6_debug.log, node7.log, node7_debug.log,
> node8.log, node8_debug.log, node9.log, node9_debug.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_large_dtest/10/testReport/consistency_test/TestAvailability/test_network_topology_strategy_each_quorum
> Failed on CassCI build trunk_large_dtest #10
> Logs are attached.
> {code}
> Stacktrace
> File "/usr/lib/python2.7/unittest/case.py", line 358, in run
> self.tearDown()
> File "/home/automaton/cassandra-dtest/dtest.py", line 719, in tearDown
> raise AssertionError('Unexpected error in log, see stdout')
> Standard Output
> Unexpected error in node3 log, error:
> ERROR [SharedPool-Worker-1] 2016-06-03 14:25:27,460 Keyspace.java:504 -
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7
> (mytestks.users)
> ERROR [SharedPool-Worker-2] 2016-06-03 14:25:27,460 Keyspace.java:504 -
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7
> (mytestks.users)
> ERROR [SharedPool-Worker-3] 2016-06-03 14:25:27,462 Keyspace.java:504 -
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7
> (mytestks.users)
> ERROR [SharedPool-Worker-2] 2016-06-03 14:25:27,464 Keyspace.java:504 -
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7
> (mytestks.users)
> ERROR [SharedPool-Worker-3] 2016-06-03 14:25:27,464 Keyspace.java:504 -
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7
> (mytestks.users)
> ERROR [SharedPool-Worker-1] 2016-06-03 14:25:27,465 Keyspace.java:504 -
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7
> (mytestks.users)
> ERROR [SharedPool-Worker-4] 2016-06-03 14:25:27,465 Keyspace.java:504 -
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7
> (mytestks.users)
> ERROR [SharedPool-Worker-5] 2016-06-03 14:25:27,465 Keyspace.java:504 -
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7
> (mytestks.users)
> ERROR [SharedPool-Worker-7] 2016-06-03 14:25:27,465 Keyspace.java:504 -
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7
> (mytestks.users)
> ERROR [SharedPool-Worker-6] 2016-06-03 14:25:27,465 Keyspace.java:504 -
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7
> (mytestks.users)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
