[
https://issues.apache.org/jira/browse/CASSANDRA-12457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426078#comment-15426078
]
Stefania commented on CASSANDRA-12457:
--------------------------------------
Reproduced in 2.2
[here|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-dtest-multiplex/18/testReport/junit/node_0_iter_024.upgrade_tests.cql_tests/TestCQLNodes2RF1_Upgrade_current_2_2_x_To_indev_3_x/bug_5732_test/]
with a slightly different allocation call stack.
In 2.1 the leak should be caused by shadow sstables not being released:
|2.1|[patch|https://github.com/stef1927/cassandra/commits/12457-2.1]|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-12457-2.1-testall/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-12457-2.1-dtest/]|[multiplexed|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-dtest-multiplex/19/]|
> dtest failure in
> upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_current_2_1_x_To_indev_2_2_x.bug_5732_test
> ------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-12457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12457
> Project: Cassandra
> Issue Type: Bug
> Reporter: Craig Kodman
> Assignee: Stefania
> Labels: dtest
> Fix For: 2.2.x
>
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.2_dtest_upgrade/16/testReport/upgrade_tests.cql_tests/TestCQLNodes2RF1_Upgrade_current_2_1_x_To_indev_2_2_x/bug_5732_test
> {code}
> Stacktrace
> File "/usr/lib/python2.7/unittest/case.py", line 358, in run
> self.tearDown()
> File "/home/automaton/cassandra-dtest/upgrade_tests/upgrade_base.py", line
> 216, in tearDown
> super(UpgradeTester, self).tearDown()
> File "/home/automaton/cassandra-dtest/dtest.py", line 666, in tearDown
> raise AssertionError('Unexpected error in log, see stdout')
> "Unexpected error in log, see stdout\n-------------------- >> begin captured
> logging << --------------------\ndtest: DEBUG: Upgrade test beginning,
> setting CASSANDRA_VERSION to 2.1.15, and jdk to 8. (Prior values will be
> restored after test).\ndtest: DEBUG: cluster ccm directory:
> /mnt/tmp/dtest-D8UF3i\ndtest: DEBUG: Done setting configuration options:\n{
> 'initial_token': None,\n 'num_tokens': '32',\n 'phi_convict_threshold':
> 5,\n 'range_request_timeout_in_ms': 10000,\n
> 'read_request_timeout_in_ms': 10000,\n 'request_timeout_in_ms': 10000,\n
> 'truncate_request_timeout_in_ms': 10000,\n 'write_request_timeout_in_ms':
> 10000}\ndtest: DEBUG: [[Row(table_name=u'ks', index_name=u'test.testindex')],
> [Row(table_name=u'ks', index_name=u'test.testindex')]]\ndtest: DEBUG:
> upgrading node1 to git:91f7387e1f785b18321777311a5c3416af0663c2\nccm: INFO:
> Fetching Cassandra updates...\ndtest: DEBUG: Querying upgraded node\ndtest:
> DEBUG: Querying old node\ndtest: DEBUG: removing ccm cluster test at:
> /mnt/tmp/dtest-D8UF3i\ndtest: DEBUG: clearing ssl stores from
> [/mnt/tmp/dtest-D8UF3i] directory\n--------------------- >> end captured
> logging << ---------------------"
> {code}
> {code}
> Standard Output
> http://git-wip-us.apache.org/repos/asf/cassandra.git
> git:91f7387e1f785b18321777311a5c3416af0663c2
> Unexpected error in node1 log, error:
> ERROR [Reference-Reaper:1] 2016-08-13 01:34:34,581 Ref.java:199 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@73deb57f) to class
> org.apache.cassandra.io.sstable.SSTableReader$DescriptorTypeTidy@2098812276:/mnt/tmp/dtest-D8UF3i/test/node1/data1/system/schema_columns-296e9c049bec3085827dc17d3df2122a/system-schema_columns-ka-4
> was not released before the reference was garbage collected
> Unexpected error in node1 log, error:
> ERROR [Reference-Reaper:1] 2016-08-13 01:34:34,581 Ref.java:199 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@7926de0f) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@1009016655:[[OffHeapBitSet]]
> was not released before the reference was garbage collected
> Unexpected error in node1 log, error:
> ERROR [Reference-Reaper:1] 2016-08-13 01:34:34,581 Ref.java:199 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@3a5760f9) to class
> org.apache.cassandra.io.util.MmappedSegmentedFile$Cleanup@223486002:/mnt/tmp/dtest-D8UF3i/test/node1/data0/system/schema_columns-296e9c049bec3085827dc17d3df2122a/system-schema_columns-ka-3-Index.db
> was not released before the reference was garbage collected
> Unexpected error in node1 log, error:
> ERROR [Reference-Reaper:1] 2016-08-13 01:34:34,582 Ref.java:199 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@42cb4131) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@1544265728:[Memory@[0..4),
> Memory@[0..a)] was not released before the reference was garbage collected
> Unexpected error in node1 log, error:
> ERROR [Reference-Reaper:1] 2016-08-13 01:34:34,582 Ref.java:199 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@5dda43d0) to class
> org.apache.cassandra.io.util.MmappedSegmentedFile$Cleanup@1100327913:/mnt/tmp/dtest-D8UF3i/test/node1/data1/system/schema_columns-296e9c049bec3085827dc17d3df2122a/system-schema_columns-ka-4-Index.db
> was not released before the reference was garbage collected
> Unexpected error in node1 log, error:
> ERROR [Reference-Reaper:1] 2016-08-13 01:34:34,582 Ref.java:199 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@59cfa823) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@1480923322:[Memory@[0..4),
> Memory@[0..a)] was not released before the reference was garbage collected
> Unexpected error in node1 log, error:
> ERROR [Reference-Reaper:1] 2016-08-13 01:34:34,601 Ref.java:199 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@570e14a1) to class
> org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Cleanup@992487242:/mnt/tmp/dtest-D8UF3i/test/node1/data1/system/schema_columns-296e9c049bec3085827dc17d3df2122a/system-schema_columns-ka-4-Data.db
> was not released before the reference was garbage collected
> Unexpected error in node1 log, error:
> ERROR [Reference-Reaper:1] 2016-08-13 01:34:34,602 Ref.java:199 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@1f021ebc) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@1878148398:[Memory@[0..4),
> Memory@[0..e)] was not released before the reference was garbage collected
> Unexpected error in node1 log, error:
> ERROR [Reference-Reaper:1] 2016-08-13 01:34:34,604 Ref.java:199 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@48feef6d) to class
> org.apache.cassandra.io.sstable.SSTableReader$DescriptorTypeTidy@848724815:/mnt/tmp/dtest-D8UF3i/test/node1/data0/system/schema_columns-296e9c049bec3085827dc17d3df2122a/system-schema_columns-ka-3
> was not released before the reference was garbage collected
> Unexpected error in node1 log, error:
> ERROR [Reference-Reaper:1] 2016-08-13 01:34:34,605 Ref.java:199 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@3dee8c5f) to class
> org.apache.cassandra.io.sstable.SSTableReader$DescriptorTypeTidy@1078490617:/mnt/tmp/dtest-D8UF3i/test/node1/data1/system/schema_columns-296e9c049bec3085827dc17d3df2122a/system-schema_columns-ka-2
> was not released before the reference was garbage collected
> Unexpected error in node1 log, error:
> ERROR [Reference-Reaper:1] 2016-08-13 01:34:34,614 Ref.java:199 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@7f726f1a) to class
> org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Cleanup@2037913408:/mnt/tmp/dtest-D8UF3i/test/node1/data0/system/schema_columns-296e9c049bec3085827dc17d3df2122a/system-schema_columns-ka-3-Data.db
> was not released before the reference was garbage collected
> Unexpected error in node1 log, error:
> ERROR [Reference-Reaper:1] 2016-08-13 01:34:34,615 Ref.java:199 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@303df044) to class
> org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Cleanup@861514759:/mnt/tmp/dtest-D8UF3i/test/node1/data1/system/schema_columns-296e9c049bec3085827dc17d3df2122a/system-schema_columns-ka-2-Data.db
> was not released before the reference was garbage collected
> Unexpected error in node1 log, error:
> ERROR [Reference-Reaper:1] 2016-08-13 01:34:34,616 Ref.java:199 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@5fbf0fc9) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@1715786089:[[OffHeapBitSet]]
> was not released before the reference was garbage collected
> Unexpected error in node1 log, error:
> ERROR [Reference-Reaper:1] 2016-08-13 01:34:34,616 Ref.java:199 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@3924b235) to class
> org.apache.cassandra.io.util.MmappedSegmentedFile$Cleanup@1197672578:/mnt/tmp/dtest-D8UF3i/test/node1/data1/system/schema_columns-296e9c049bec3085827dc17d3df2122a/system-schema_columns-ka-2-Index.db
> was not released before the reference was garbage collected
> Unexpected error in node1 log, error:
> ERROR [Reference-Reaper:1] 2016-08-13 01:34:34,616 Ref.java:199 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@600596e0) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@545967120:[[OffHeapBitSet]]
> was not released before the reference was garbage collected
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
