[
https://issues.apache.org/jira/browse/CASSANDRA-7186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14233268#comment-14233268
]
Alexander Bulaev commented on CASSANDRA-7186:
---------------------------------------------
1. No, test clusters are not under heavy load, they probably get 1-2 RPS. By
the way, even the production cluster does not seem to be overloaded (LA is up
to 3.5 in rush hour on 32-core systems, which seems perfectly OK). At peak we
are having ~250 RPS writes and ~100 RPS reads in production environment.
Config file is available here:
https://www.dropbox.com/s/x5xhtkox7sj25u6/cassandra.yaml?dl=0
Unfortunately the logs were rotated, but when I was watching them when the
issue had occurred, I could not find any suspicious messages.
2. Yes, we are using NTP and clocks are in perfect sync.
> alter table add column not always propogating
> ---------------------------------------------
>
> Key: CASSANDRA-7186
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7186
> Project: Cassandra
> Issue Type: Bug
> Reporter: Martin Meyer
> Assignee: Philip Thompson
> Fix For: 2.0.12
>
>
> I've been many times in Cassandra 2.0.6 that adding columns to existing
> tables seems to not fully propagate to our entire cluster. We add an extra
> column to various tables maybe 0-2 times a week, and so far many of these
> ALTERs have resulted in at least one node showing the old table description a
> pretty long time (~30 mins) after the original ALTER command was issued.
> We originally identified this issue when a connected clients would complain
> that a column it issued a SELECT for wasn't a known column, at which point we
> have to ask each node to describe the most recently altered table. One of
> them will not know about the newly added field. Issuing the original ALTER
> statement on that node makes everything work correctly.
> We have seen this issue on multiple tables (we don't always alter the same
> one). It has affected various nodes in the cluster (not always the same one
> is not getting the mutation propagated). No new nodes have been added to the
> cluster recently. All nodes are homogenous (hardware and software), running
> 2.0.6. We don't see any particular errors or exceptions on the node that
> didn't get the schema update, only the later error from a Java client about
> asking for an unknown column in a SELECT. We have to check each node manually
> to find the offender. The tables he have seen this on are under fairly heavy
> read and write load, but we haven't altered any tables that are not, so that
> might not be important.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
