[
https://issues.apache.org/jira/browse/CASSANDRA-3681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13179576#comment-13179576
]
Brandon Williams commented on CASSANDRA-3681:
---------------------------------------------
The problem here is that now the schema check comes before the FD check:
{noformat}
ERROR 15:45:37,154 Fatal exception in thread Thread[HintedHandoff:1,1,main]
java.lang.RuntimeException: Didin't receive gossiped schema from /10.179.64.227
in 60000ms
at
org.apache.cassandra.db.HintedHandOffManager.waitForSchemaAgreement(HintedHandOffManager.java:210)
at
org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:239)
at
org.apache.cassandra.db.HintedHandOffManager.access$200(HintedHandOffManager.java:84)
at
org.apache.cassandra.db.HintedHandOffManager$3.runMayThrow(HintedHandOffManager.java:385)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
{noformat}
I added another FD check for this in the patch on CASSANDRA-3677 (which isn't
the problem there) but something similar should work.
> Multiple threads can attempt hint handoff to the same target
> ------------------------------------------------------------
>
> Key: CASSANDRA-3681
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3681
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Priority: Minor
> Labels: hintedhandoff
> Fix For: 1.0.7
>
> Attachments: 3681.txt
>
>
> HintedHandOffManager attempts to prevent multiple threads sending hints to
> the same target with the queuedDeliveries set, but the code is buggy. If two
> handoffs *do* occur concurrently, the second thread can use an arbitrarily
> large amount of memory skipping tombstones when it starts paging from the
> beginning of the hint row, looking for the first live hint. (This is not a
> problem with a single thread, since it always pages starting with the
> last-seen hint column name, effectively skipping the tombstones. Then it
> compacts when it's done.)
> Technically this bug is present in all older Cassandra releases, but it only
> causes problems in 1.0.x since the hint rows tend to be much larger (since
> there is one hint per write containing the entire mutation, instead of just
> one per row consisting of just the key).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
