Hi all,
I was wondering if anybody had seen the following behaviour before and
how we might detect it and keep the application running.
We have a 6 node cluster. It seems that one of these nodes forgot about
all but one of the application column families - possibly after a
restart. Then, when our application connects using Hector, it can't find
any data so gives back an exception.
I'm currently running nodetool repair on one of the *other* nodes which
is taking a very long time to complete. (35mins and counting a load of <9MB)
The logs from the failing node say:
INFO [MemoryMeter:1] 2012-08-23 14:59:14,807 Memtable.java (line 213)
CFS(Keyspace='system', ColumnFamily='HintsColumnFamily') liveRatio is
1.1219167666485013 (just-counted was 1.0). calculation took 28ms for
1252 columns
INFO [main] 2012-08-23 14:59:14,949 CommitLogReplayer.java (line 272)
Finished reading /var/lib/cassandra/commitlog/CommitLog-22654969122258
24.log
INFO [main] 2012-08-23 14:59:14,950 CommitLogReplayer.java (line 103)
Skipped 8216 mutations from unknown (probably removed) CF with id 1016
INFO [main] 2012-08-23 14:59:14,950 CommitLogReplayer.java (line 103)
Skipped 3013 mutations from unknown (probably removed) CF with id 1017
... and so on.
Hector is saying:
InvalidRequestException(why:unconfigured columnfamily user_conversations)
Thanks for any comments or advice,
Edward
--
Edward Sargisson
senior java developer
Global Relay
edward.sargis...@globalrelay.net <mailto:edward.sargis...@globalrelay.net>
*866.484.6630*
New York | Chicago | Vancouver | London (+44.0800.032.9829) | Singapore
(+65.3158.1301)
Global Relay Archive supports email, instant messaging, BlackBerry,
Bloomberg, Thomson Reuters, Pivot, YellowJacket, LinkedIn, Twitter,
Facebook and more.
Ask about *Global Relay Message*
<http://www.globalrelay.com/services/message>*--- *The Future of
Collaboration in the Financial Services World
*
*All email sent to or from this address will be retained by Global
Relay's email archiving system. This message is intended only for the
use of the individual or entity to which it is addressed, and may
contain information that is privileged, confidential, and exempt from
disclosure under applicable law. Global Relay will not be liable for
any compliance or technical information provided herein. All trademarks
are the property of their respective owners.
