[ 
https://issues.apache.org/jira/browse/CASSANDRA-11090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137155#comment-15137155
 ] 

Blake Eggleston commented on CASSANDRA-11090:
---------------------------------------------

Ok, looks like the crc problem is gone in the current 3.0 branch.

The main issue I'm seeing though, is that any hint activity starts an infinite 
loop of empty hint file creation / processing. In the script I linked in the 
description, some data is inserted while there's a node down, creating a hints 
file. When the node is brought up, the hints are transmitted, as you'd expect. 
After that, every 10 seconds, another hints file (empty this time) is created 
for that node, and processed / deleted.

It's less of an issue if the crc problem is fixed, but will still create a lot 
of noise in the logs for moderately sized clusters.

> Hinted Handoff loop
> -------------------
>
>                 Key: CASSANDRA-11090
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-11090
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Blake Eggleston
>            Assignee: Aleksey Yeschenko
>             Fix For: 3.0.x, 3.x
>
>
> After the hints executor finishes sending hints, another hints file is 
> created for the same host, which is then processed on the next go around. 
> This continues indefinitely. The new hint files are empty, so there shouldn't 
> be any network traffic. However, there's still unnecessary hint activity, and 
> the hint crc file isn't deleted in the 3.0 branch after hints are dispatched 
> (but is in trunk), so any hint activity will trigger ~8640 files being 
> created per day until the node is restarted. Restarting the node fixes the 
> problem, after the existing hint files are processed. 
> This can be duplicated on cassandra-3.0 to trunk with this script: 
> https://gist.github.com/bdeggleston/13fbb9e70c0c0bd277c7



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to