[
https://issues.apache.org/jira/browse/CASSANDRA-9846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14644003#comment-14644003
]
Will Zhang commented on CASSANDRA-9846:
---------------------------------------
FYI - upgraded to 2.2.0. Have not seen this problem so far and server running
fine. No error in log.
> CassandraDaemon fails to start due to FileNotFoundException
> -----------------------------------------------------------
>
> Key: CASSANDRA-9846
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9846
> Project: Cassandra
> Issue Type: Bug
> Environment: Windows 7 64bit, 4-core, 24GB memory. Running single
> cluster.
> Java 1.8.0_45, python 2.7.10
> Reporter: Will Zhang
> Attachments: system.2.1.8.log
>
>
> Cassandra 2.1.8
> Server fails to start after seeing a `FileNotFoundException` in the logs.
> Before upgrading to `2.1.8`, I was running `2.1.5`. I ran the following
> before shutting down `2.1.5`:
> nodetool flush
> nodetool drain
> notetool repair
> When I tred to start 2.1.8 after a reboot, the following error is observed in
> the system.log file. Server would not start after reach this point. I also
> attached the full error log which shows two start attempts.
> When I reverted back to 2.1.5, server started just fine, with no error in log.
> I saw the following issue reported but that was for 2.0.9. However, I have
> not seen this error in 2.1.3, 2.1.4 or 2.1.5.
> https://issues.apache.org/jira/browse/CASSANDRA-7780
> Error log:
> {code:java}
> INFO [main] 2015-07-18 21:30:05,582 YamlConfigurationLoader.java:135 - Node
> configuration:[authenticator=AllowAllAuthenticator;
> authorizer=AllowAllAuthorizer; auto_snapshot=true;
> batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024;
> cas_contention_timeout_in_ms=5000; client_encryption_options=<REDACTED>;
> cluster_name=zhangwe; column_index_size_in_kb=64; commit_failure_policy=stop;
> commitlog_directory=z:/cassandra/commitlog; commitlog_segment_size_in_mb=32;
> commitlog_sync=periodic; commitlog_sync_period_in_ms=10000;
> compaction_throughput_mb_per_sec=16; concurrent_counter_writes=32;
> concurrent_reads=32; concurrent_writes=32; counter_cache_save_period=7200;
> counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=40000;
> cross_node_timeout=false; data_file_directories=[z:/cassandra/data];
> disk_failure_policy=stop; dynamic_snitch_badness_threshold=0.1;
> dynamic_snitch_reset_interval_in_ms=600000;
> dynamic_snitch_update_interval_in_ms=100; endpoint_snitch=SimpleSnitch;
> hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024;
> incremental_backups=false; index_summary_capacity_in_mb=null;
> index_summary_resize_interval_in_minutes=60; inter_dc_tcp_nodelay=false;
> internode_compression=all; key_cache_save_period=14400;
> key_cache_size_in_mb=null; listen_address=localhost;
> max_hint_window_in_ms=10800000; max_hints_delivery_threads=2;
> memtable_allocation_type=heap_buffers; native_transport_port=9042;
> num_tokens=256; partitioner=org.apache.cassandra.dht.Murmur3Partitioner;
> permissions_validity_in_ms=2000; range_request_timeout_in_ms=40000;
> read_request_timeout_in_ms=30000;
> request_scheduler=org.apache.cassandra.scheduler.NoScheduler;
> request_timeout_in_ms=50000; row_cache_save_period=0; row_cache_size_in_mb=0;
> rpc_address=localhost; rpc_keepalive=true; rpc_port=9160;
> rpc_server_type=sync; saved_caches_directory=z:/cassandra/saved_caches;
> seed_provider=[{class_name=org.apache.cassandra.locator.SimpleSeedProvider,
> parameters=[{seeds=127.0.0.1}]}]; server_encryption_options=<REDACTED>;
> snapshot_before_compaction=false; ssl_storage_port=7001;
> sstable_preemptive_open_interval_in_mb=50; start_native_transport=true;
> start_rpc=true; storage_port=7000; thrift_framed_transport_size_in_mb=15;
> tombstone_failure_threshold=100000; tombstone_warn_threshold=1000;
> trickle_fsync=false; trickle_fsync_interval_in_kb=10240;
> truncate_request_timeout_in_ms=90000; write_request_timeout_in_ms=30000]
> INFO [main] 2015-07-18 21:30:05,683 MessagingService.java:480 - Starting
> Messaging Service on port 7000
> ERROR [NonPeriodicTasks:1] 2015-07-18 21:30:05,685
> SSTableDeletingTask.java:85 - Unable to delete
> z:\cassandra\data\system\compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca\system-compaction_history-tmplink-ka-1539-Data.db
> (it will be removed on server restart; we'll also retry after GC)
> ERROR [main] 2015-07-18 21:30:05,747 CassandraDaemon.java:541 - Exception
> encountered during startup
> java.lang.RuntimeException: java.io.FileNotFoundException:
> z:\cassandra\data\system\local-7ad54392bcdd35a684174e047860b377\system-local-tmplink-ka-338-Data.db
> (The system cannot find the file specified)
> at
> org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:52)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.io.util.CompressedPoolingSegmentedFile.createPooledReader(CompressedPoolingSegmentedFile.java:95)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.io.util.PoolingSegmentedFile.getSegment(PoolingSegmentedFile.java:62)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.io.sstable.SSTableReader.getFileDataInput(SSTableReader.java:1822)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.db.columniterator.SimpleSliceReader.<init>(SimpleSliceReader.java:57)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.db.columniterator.SSTableSliceIterator.createReader(SSTableSliceIterator.java:65)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.db.columniterator.SSTableSliceIterator.<init>(SSTableSliceIterator.java:42)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:246)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:62)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:270)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:65)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1967)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1810)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:357)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:85)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.cql3.statements.SelectStatement.readLocally(SelectStatement.java:309)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.cql3.statements.SelectStatement.executeInternal(SelectStatement.java:328)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.cql3.statements.SelectStatement.executeInternal(SelectStatement.java:67)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.cql3.QueryProcessor.executeInternal(QueryProcessor.java:317)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.db.SystemKeyspace.getBootstrapState(SystemKeyspace.java:680)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.db.SystemKeyspace.bootstrapInProgress(SystemKeyspace.java:695)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:823)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:723)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:605)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378)
> [apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:524)
> [apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:613)
> [apache-cassandra-2.1.8.jar:2.1.8]
> Caused by: java.io.FileNotFoundException:
> z:\cassandra\data\system\local-7ad54392bcdd35a684174e047860b377\system-local-tmplink-ka-338-Data.db
> (The system cannot find the file specified)
> at java.io.RandomAccessFile.open(Native Method) ~[na:1.8.0_11]
> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:236)
> ~[na:1.8.0_11]
> at
> org.apache.cassandra.io.util.RandomAccessReader.<init>(RandomAccessReader.java:64)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.io.compress.CompressedRandomAccessReader.<init>(CompressedRandomAccessReader.java:70)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:48)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> ... 26 common frames omitted
> INFO [CompactionExecutor:1] 2015-07-18 21:30:05,819 CompactionTask.java:274
> - Compacted 4 sstables to
> [z:\cassandra\data\system\compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca\system-compaction_history-ka-1539,].
> 13,359 bytes to 12,686 (~94% of original) in 565ms = 0.021413MB/s. 134
> total partitions merged to 134. Partition merge counts were {1:134, }
> INFO [StorageServiceShutdownHook] 2015-07-18 21:30:05,833 Gossiper.java:1410
> - Announcing shutdown
> INFO [CompactionExecutor:2] 2015-07-18 21:30:06,023 CompactionTask.java:274
> - Compacted 4 sstables to
> [z:\cassandra\data\system\local-7ad54392bcdd35a684174e047860b377\system-local-ka-338,].
> 11,594 bytes to 5,856 (~50% of original) in 433ms = 0.012898MB/s. 4 total
> partitions merged to 1. Partition merge counts were {4:1, }
> ERROR [NonPeriodicTasks:1] 2015-07-18 21:30:06,100
> SSTableDeletingTask.java:85 - Unable to delete
> z:\cassandra\data\system\local-7ad54392bcdd35a684174e047860b377\system-local-tmplink-ka-338-Data.db
> (it will be removed on server restart; we'll also retry after GC)
> ERROR [NonPeriodicTasks:1] 2015-07-18 21:30:06,560
> SSTableDeletingTask.java:85 - Unable to delete
> z:\cassandra\data\system\compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca\system-compaction_history-tmplink-ka-1539-Data.db
> (it will be removed on server restart; we'll also retry after GC)
> ERROR [NonPeriodicTasks:1] 2015-07-18 21:30:06,564
> SSTableDeletingTask.java:85 - Unable to delete
> z:\cassandra\data\system\local-7ad54392bcdd35a684174e047860b377\system-local-tmplink-ka-338-Data.db
> (it will be removed on server restart; we'll also retry after GC)
> INFO [StorageServiceShutdownHook] 2015-07-18 21:30:07,866
> MessagingService.java:708 - Waiting for messaging service to quiesce
> INFO [ACCEPT-localhost/127.0.0.1] 2015-07-18 21:30:07,867
> MessagingService.java:958 - MessagingService has terminated the accept()
> thread
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
