[
https://issues.apache.org/jira/browse/CASSANDRA-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13157403#comment-13157403
]
Jonathan Ellis commented on CASSANDRA-3533:
-------------------------------------------
Alex Feinberg from Voldemort says:
bq. We had this situation by accident in production, when nodes in the other
datacenter were firewalled off from clients in one cluster. The way we deal
with it is, our failure detector is local to each client and has a thread which
keeps pinging each node it marked down initially to see if it came back up.
ThreadholdFailureDetector, which inherits from AsyncRecoveryFailureDetector.
> TimeoutException when there is a firewall issue.
> ------------------------------------------------
>
> Key: CASSANDRA-3533
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3533
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 1.0.4
> Reporter: Vijay
> Priority: Minor
>
> When one node in the cluster is not able to talk to the other DC/RAC due to
> firewall or network related issue (StorageProxy calls fail), and the nodes
> are NOT marked down because at least one node in the cluster can talk to the
> other DC/RAC, we get timeoutException instead of throwing a
> unavailableException.
> The problem with this:
> 1) It is hard to monitor/identify these errors.
> 2) It is hard to diffrentiate from the client if the node being bad vs a bad
> query.
> 3) when this issue happens we have to wait for at-least the RPC timeout time
> to know that the query wont succeed.
> Possible Solution: when marking a node down we might want to check if the
> node is actually alive by trying to communicate to it? So we can be sure that
> the node is actually alive.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
