[
https://issues.apache.org/jira/browse/CASSANDRA-5025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510585#comment-13510585
]
Chris Herron commented on CASSANDRA-5025:
-----------------------------------------
Clarifying for anyone else who encounters this issue:
* This problem was introduced in CASSANDRA-3931
* For use cases that involve creation/update/deletion of multiple keyspaces or
column families, the symptom will be increasingly slow schema migrations as the
KS/CF population grows. Depending on client RPC timeout config, schema change
requests may fail.
* In a test environment running stock C* 1.1.7, for a test that creates new CFs
in sequence, we see the following CF creation times:
** Empty cluster: sub-second
** 200+ CFs: 15s ave.
** 400+ CFs: 30s+ with eventual failure due to 30s client side (Hector) RPC
timeout.
* In the same test environment running 1.1.7 patched with 5025.txt:
** For the first 60s duration of the test, CF creation times are sub-second
** At 60s, the delayed rectifySchema migration calls kick in and creation times
drop to 50s+ (including waits for schema agreement) with eventual failure due
to 30s client side RPC timeout.
> Schema push/pull race
> ---------------------
>
> Key: CASSANDRA-5025
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5025
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.1.0
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 1.1.8
>
> Attachments: 5025.txt
>
>
> When a schema change is made, the coordinator pushes the delta to the other
> nodes in the cluster. This is more efficient than sending the entire schema.
> But the coordinator also announces the new schema version, so the other
> nodes' reception of the new version races with processing the delta, and
> usually seeing the new schema wins. So the other nodes also issue a pull to
> the coordinator for the entire schema.
> Thus, schema changes tend to become O(n) in the number of KS and CF present.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
