[
https://issues.apache.org/jira/browse/CASSANDRA-11090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429960#comment-15429960
]
Cameron Zemek commented on CASSANDRA-11090:
-------------------------------------------
In both 3.5 and 3.7 we seeing hinted handoff loop
Aug 04 07:20:33 ip-10-222-104-29.ec2.internal cassandra[31062]: INFO
o.a.c.hints.HintsDispatchExecutor Finished hinted handoff of file
4566c9ad-69df-45fd-84d2-6386072d3efa-1470295200416-1.hints to endpoint
4566c9ad-69df-45fd-84d2-6386072d3efa
Aug 04 07:21:03 ip-10-222-104-29.ec2.internal cassandra[31062]: INFO
o.apache.cassandra.hints.HintsStore Deleted hint file
4566c9ad-69df-45fd-84d2-6386072d3efa-1470295230416-1.hints
Aug 04 07:21:03 ip-10-222-104-29.ec2.internal cassandra[31062]: INFO
o.a.c.hints.HintsDispatchExecutor Finished hinted handoff of file
4566c9ad-69df-45fd-84d2-6386072d3efa-1470295230416-1.hints to endpoint
4566c9ad-69df-45fd-84d2-6386072d3efa
Aug 04 07:21:33 ip-10-222-104-29.ec2.internal cassandra[31062]: INFO
o.apache.cassandra.hints.HintsStore Deleted hint file
4566c9ad-69df-45fd-84d2-6386072d3efa-1470295260416-1.hints
and it just keeps repeating this. Have to rolling restart the cluster to
resolve. Waiting for it to happen again to collect more information on the
cause for this loop.
> Hinted Handoff loop
> -------------------
>
> Key: CASSANDRA-11090
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11090
> Project: Cassandra
> Issue Type: Bug
> Reporter: Blake Eggleston
> Assignee: Aleksey Yeschenko
> Priority: Minor
> Fix For: 3.0.4, 3.4
>
>
> After the hints executor finishes sending hints, another hints file is
> created for the same host, which is then processed on the next go around.
> This continues indefinitely. The new hint files are empty, so there shouldn't
> be any network traffic. However, there's still unnecessary hint activity, and
> the hint crc file isn't deleted in the 3.0 branch after hints are dispatched
> (but is in trunk), so any hint activity will trigger ~8640 files being
> created per day until the node is restarted. Restarting the node fixes the
> problem, after the existing hint files are processed.
> This can be duplicated on cassandra-3.0 to trunk with this script:
> https://gist.github.com/bdeggleston/13fbb9e70c0c0bd277c7
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
