Hi Aaron,
The error is being reported on all 4 nodes. I have confirmed (for my
own sanity) that each node is running: ReleaseVersion: 0.8.0
I can reproduce the error on any node by trailing
cassandra/logs/system.log and running "nodetool repair"
INFO [manual-repair-1c6b33bc-ef14-4ec8-94f6-f1464ec8bdec] 2011-06-13
21:28:39,877 AntiEntropyService.java (line 177) Excluding
/10.128.34.18 from repair because it is on version 0.7 or sooner. You
should consider updating this node before running repair again.
ERROR [manual-repair-1c6b33bc-ef14-4ec8-94f6-f1464ec8bdec] 2011-06-13
21:28:39,877 AbstractCassandraDaemon.java (line 113) Fatal exception
in thread Thread[manual-repair-1c6b33bc-ef14-4ec8-94f6-f1464ec8bdec,5,RMI
Runtime]
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
at java.util.HashMap$KeyIterator.next(HashMap.java:828)
at
org.apache.cassandra.service.AntiEntropyService.getNeighbors(AntiEntropyService.java:173)
at
org.apache.cassandra.service.AntiEntropyService$RepairSession.run(AntiEntropyService.java:776)
When I run nodetool ring, the ring looks balanced and nothing out of sorts.
I also have this set up with RF=3 on 4 nodes ... but repair was
working fine prior to the 0.8.0 upgrade.
Are there any special commands I need to run? I've tried scrub,
cleanup, flush too ... still, repair gives the same issues.
-- I have stopped one of the nodes and started it. Issue still
persists. I stop another node that is reported in the logs (like .18
above) and start it ... run repair again ... issue is persisted to the
log file still.
-sd
On Mon, Jun 13, 2011 at 3:02 PM, aaron morton <[email protected]> wrote:
> You can double check with node tool e.g.
>
> $ ./bin/nodetool -h localhost version
> ReleaseVersion: 0.8.0-SNAPSHOT
>
> This error is about the internode wire protocol one node thinks another is
> using. Not sure how it could get confused, does it go away if you restart the
> node that logged the error ?
>
> Cheers
>
> -----------------
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 13 Jun 2011, at 06:19, Sasha Dolgy wrote:
>
>> Hi Everyone,
>>
>> Last week, upgraded all 4 nodes to apache-cassandra-0.8.0 .. no
>> issues. Trolling the logs today, I find messages like this on all
>> four nodes:
>>
>> INFO [manual-repair-0b61c9e2-3593-4633-a80f-b6ca52cfe948] 2011-06-13
>> 02:16:45,978 AntiEntropyService.java (line 177) Excluding
>> /10.128.34.18 from repair because it is on version 0.7 or sooner. You
>> should consider updating this node before running repair again.
>>
>> Maybe it would be nice to have the version of all nodes print in
>> nodetool ring ? I don't think I'm crazy though ... have manually
>> checked all are on 0.8.0
>>
>>
>> --
>> Sasha Dolgy
>> [email protected]
>
>
--
Sasha Dolgy
[email protected]
