[
https://issues.apache.org/jira/browse/CASSANDRA-7199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Shuler updated CASSANDRA-7199:
--------------------------------------
Attachment: jenkins-scratch-2.1_dtest-failed-snapshot_test-dtestdir.tar.gz
dtest DEBUG log (last line created when SIGKILL was sent at job abort):
{noformat}
11:57:44,461 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG cluster ccm directory: /tmp/dtest-HHwzRI
11:57:44,600 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG tmp_commitlog: /tmp/tmp4PKZDL
11:57:49,723 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG Writing first 30,000 rows...
11:57:59,958 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG Making snapshot....
11:58:02,171 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG Running snapshot cmd: snapshot ks -cf cf -t basic
11:58:03,559 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG snapshot_dir is :
/tmp/dtest-HHwzRI/test/node1/data/ks/cf-2494e1a0d77111e393f77ff88ef322c6/snapshots/basic
11:58:03,559 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG snapshot copy is : /tmp/tmpgTsloD
11:58:03,637 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG Writing second 30,000 rows...
11:58:22,562 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG Writing final 5,000 rows...
11:58:25,215 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG node1 commitlog dir: /tmp/dtest-HHwzRI/test/node1/commitlogs
11:58:34,394 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG removing ccm cluster test at: /tmp/dtest-HHwzRI
11:58:34,404 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG cluster ccm directory: /tmp/dtest-ElgmcU
11:58:39,747 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG Restoring snapshot....
11:58:45,70 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG Restarting node1..
11:58:59,783 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG removing snapshot_dir: /tmp/tmpgTsloD
11:58:59,785 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG removing tmp_commitlog: /tmp/tmp4PKZDL
11:58:59,787 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG removing ccm cluster test at: /tmp/dtest-ElgmcU
11:58:59,899 dtest
snapshot_test.TestArchiveCommitlog.test_archive_commitlogtest_archive_commitlog
DEBUG cluster ccm directory: /tmp/dtest-se6EyY
11:59:00,35 dtest
snapshot_test.TestArchiveCommitlog.test_archive_commitlogtest_archive_commitlog
DEBUG tmp_commitlog: /tmp/tmp6c0qNr
11:59:05,146 dtest
snapshot_test.TestArchiveCommitlog.test_archive_commitlogtest_archive_commitlog
DEBUG Writing first 30,000 rows...
11:59:15,186 dtest
snapshot_test.TestArchiveCommitlog.test_archive_commitlogtest_archive_commitlog
DEBUG Making snapshot....
12:14:15,224 dtest
snapshot_test.TestArchiveCommitlog.test_archive_commitlogtest_archive_commitlog
DEBUG Running snapshot cmd: snapshot ks -cf cf -t basic
{noformat}
ccm node1 log (full ccm node tar attached) shows:
{noformat}
INFO [MemtableFlushWriter:1] 2014-05-09 11:59:05,098 Memtable.java:365 -
Completed flushing
/tmp/dtest-se6EyY/test/node1/data/system/schema_columns-296e9c049bec3085827dc17d3df2122a/system-schema_columns-ka-2-Data.db
(306 bytes) for commitlog position ReplayPosition(segmentId=1399636743311,
position=114155)
INFO [MigrationStage:1] 2014-05-09 11:59:05,121 DefsTables.java:388 - Loading
org.apache.cassandra.config.CFMetaData@5341453d[cfId=5189aba0-d771-11e3-ba97-7ff88ef322c6,ksName=ks,cfName=cf,cfType=Standard,comparator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type),comment=,readRepairChance=0.1,dcLocalReadRepairChance=0.0,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.LongType,minCompactionThreshold=4,maxCompactionThreshold=32,columnMetadata={java.nio.HeapByteBuffer[pos=0
lim=3 cap=3]=ColumnDefinition{name=key,
type=org.apache.cassandra.db.marshal.LongType, kind=PARTITION_KEY,
componentIndex=null, indexName=null, indexType=null},
java.nio.HeapByteBuffer[pos=0 lim=3 cap=3]=ColumnDefinition{name=val,
type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, componentIndex=0,
indexName=null, indexType=null}},compactionStrategyClass=class
org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionParameters={sstable_compression=org.apache.cassandra.io.compress.LZ4Compressor},bloomFilterFpChance=0.01,memtableFlushPeriod=0,caching={"keys":"ALL",
"rows_per_partition":"NONE"},defaultTimeToLive=0,minIndexInterval=128,maxIndexInterval=2048,speculativeRetry=99.0PERCENTILE,droppedColumns={},triggers={}]
INFO [MigrationStage:1] 2014-05-09 11:59:05,125 ColumnFamilyStore.java:284 -
Initializing ks.cf
WARN [Thrift:1] 2014-05-09 11:59:05,152 FBUtilities.java:359 - Trigger
directory doesn't exist, please create it and try again.
ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2014-05-09 11:59:09,844
AbstractCommitLogService.java:125 - Commit log sync failed
org.apache.cassandra.io.FSWriteError: java.lang.IndexOutOfBoundsException
at
org.apache.cassandra.db.commitlog.CommitLogSegment.sync(CommitLogSegment.java:337)
~[main/:na]
at org.apache.cassandra.db.commitlog.CommitLog.sync(CommitLog.java:173)
~[main/:na]
at
org.apache.cassandra.db.commitlog.AbstractCommitLogService$1.run(AbstractCommitLogService.java:81)
~[main/:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_55]
Caused by: java.lang.IndexOutOfBoundsException: null
at java.nio.Buffer.checkIndex(Buffer.java:538) ~[na:1.7.0_55]
at java.nio.DirectByteBuffer.putLong(DirectByteBuffer.java:796)
~[na:1.7.0_55]
at
org.apache.cassandra.db.commitlog.CommitLogSegment.sync(CommitLogSegment.java:319)
~[main/:na]
... 3 common frames omitted
ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2014-05-09 11:59:12,516
AbstractCommitLogService.java:125 - Commit log sync failed
org.apache.cassandra.io.FSWriteError: java.lang.IndexOutOfBoundsException
at
org.apache.cassandra.db.commitlog.CommitLogSegment.sync(CommitLogSegment.java:337)
~[main/:na]
at org.apache.cassandra.db.commitlog.CommitLog.sync(CommitLog.java:173)
~[main/:na]
at
org.apache.cassandra.db.commitlog.AbstractCommitLogService$1.run(AbstractCommitLogService.java:81)
~[main/:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_55]
Caused by: java.lang.IndexOutOfBoundsException: null
at java.nio.Buffer.checkIndex(Buffer.java:538) ~[na:1.7.0_55]
at java.nio.DirectByteBuffer.putLong(DirectByteBuffer.java:796)
~[na:1.7.0_55]
at
org.apache.cassandra.db.commitlog.CommitLogSegment.sync(CommitLogSegment.java:319)
~[main/:na]
... 3 common frames omitted
ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2014-05-09 11:59:14,798
AbstractCommitLogService.java:125 - Commit log sync failed
org.apache.cassandra.io.FSWriteError: java.lang.IndexOutOfBoundsException
at
org.apache.cassandra.db.commitlog.CommitLogSegment.sync(CommitLogSegment.java:337)
~[main/:na]
<...>
{noformat}
> [dtest] snapshot_test abort logs
> --------------------------------
>
> Key: CASSANDRA-7199
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7199
> Project: Cassandra
> Issue Type: Test
> Components: Tests
> Reporter: Michael Shuler
> Assignee: Michael Shuler
> Priority: Minor
> Labels: qa-resolved
> Fix For: 2.1 rc1
>
> Attachments: 7199.txt,
> jenkins-scratch-2.1_dtest-failed-snapshot_test-dtestdir.tar.gz
>
>
> Test hung twice on 2.1 in the same manner while trying a new ccm branch as a
> scratch jenkins job
> {noformat}
> 11:57:44 dont_test_archive_commitlog (snapshot_test.TestArchiveCommitlog) ...
> Requested creating snapshot(s) for [ks] with snapshot name [basic]
> 11:58:03 Snapshot directory: basic
> 11:58:41 Established connection to initial hosts
> 11:58:41 Opening sstables and calculating sections to stream
> 11:58:41 Streaming relevant part of /tmp/tmpgTsloD/ks/cf/ks-cf-ka-1-Data.db
> to [/127.0.0.1]
> 11:58:41
> progress: [/127.0.0.1]0:1/1 100% total: 100% 0 MB/s(avg: 0 MB/s)
> progress: [/127.0.0.1]0:1/1 100% total: 100% 0 MB/s(avg: 0 MB/s)
> 11:58:42 Summary statistics:
> 11:58:42 Connections per host: : 1
> 11:58:42 Total files transferred: : 1
> 11:58:42 Total bytes transferred: : 527659
> 11:58:42 Total duration (ms): : 2384
> 11:58:42 Average transfer rate (MB/s): : 0
> 11:58:42 Peak transfer rate (MB/s): : 0
> 11:58:42
> 11:58:59 ok
> 11:58:59 test_archive_commitlog (snapshot_test.TestArchiveCommitlog) ... rm:
> cannot remove `/tmp/tmp6c0qNr/*': No such file or directory
> 12:14:15 Build timed out (after 15 minutes). Marking the build as aborted.
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
