[jira] [Comment Edited] (CASSANDRA-7186) alter table add column not always propogating
[ https://issues.apache.org/jira/browse/CASSANDRA-7186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242063#comment-15242063 ] Uttam Phalnikar edited comment on CASSANDRA-7186 at 4/14/16 10:40 PM: -- Actually restart of a node fixed problem on that node. Is there any cache flush that I need to achieve same results without restart? PS> nodetool repair system and nodetool flush system didn't help was (Author: uttam1105): Actually restart of a node fixed problem on that node. Is there any cache flush that I need to achieve same results without restart? > alter table add column not always propogating > - > > Key: CASSANDRA-7186 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7186 > Project: Cassandra > Issue Type: Bug >Reporter: Martin Meyer >Assignee: Philip Thompson > Fix For: 2.0.12 > > > I've been many times in Cassandra 2.0.6 that adding columns to existing > tables seems to not fully propagate to our entire cluster. We add an extra > column to various tables maybe 0-2 times a week, and so far many of these > ALTERs have resulted in at least one node showing the old table description a > pretty long time (~30 mins) after the original ALTER command was issued. > We originally identified this issue when a connected clients would complain > that a column it issued a SELECT for wasn't a known column, at which point we > have to ask each node to describe the most recently altered table. One of > them will not know about the newly added field. Issuing the original ALTER > statement on that node makes everything work correctly. > We have seen this issue on multiple tables (we don't always alter the same > one). It has affected various nodes in the cluster (not always the same one > is not getting the mutation propagated). No new nodes have been added to the > cluster recently. All nodes are homogenous (hardware and software), running > 2.0.6. We don't see any particular errors or exceptions on the node that > didn't get the schema update, only the later error from a Java client about > asking for an unknown column in a SELECT. We have to check each node manually > to find the offender. The tables he have seen this on are under fairly heavy > read and write load, but we haven't altered any tables that are not, so that > might not be important. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7186) alter table add column not always propogating
[ https://issues.apache.org/jira/browse/CASSANDRA-7186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242063#comment-15242063 ] Uttam Phalnikar commented on CASSANDRA-7186: Actually restart of a node fixed problem on that node. Is there any cache flush that I need to achieve same results without restart? > alter table add column not always propogating > - > > Key: CASSANDRA-7186 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7186 > Project: Cassandra > Issue Type: Bug >Reporter: Martin Meyer >Assignee: Philip Thompson > Fix For: 2.0.12 > > > I've been many times in Cassandra 2.0.6 that adding columns to existing > tables seems to not fully propagate to our entire cluster. We add an extra > column to various tables maybe 0-2 times a week, and so far many of these > ALTERs have resulted in at least one node showing the old table description a > pretty long time (~30 mins) after the original ALTER command was issued. > We originally identified this issue when a connected clients would complain > that a column it issued a SELECT for wasn't a known column, at which point we > have to ask each node to describe the most recently altered table. One of > them will not know about the newly added field. Issuing the original ALTER > statement on that node makes everything work correctly. > We have seen this issue on multiple tables (we don't always alter the same > one). It has affected various nodes in the cluster (not always the same one > is not getting the mutation propagated). No new nodes have been added to the > cluster recently. All nodes are homogenous (hardware and software), running > 2.0.6. We don't see any particular errors or exceptions on the node that > didn't get the schema update, only the later error from a Java client about > asking for an unknown column in a SELECT. We have to check each node manually > to find the offender. The tables he have seen this on are under fairly heavy > read and write load, but we haven't altered any tables that are not, so that > might not be important. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7186) alter table add column not always propogating
[
https://issues.apache.org/jira/browse/CASSANDRA-7186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241725#comment-15241725
]
Uttam Phalnikar commented on CASSANDRA-7186:
We are experiencing similar issue intermittently. It usually happens when table
has some data (m+ records)
Steps to reproduce:
- Alter table add column
- nodetool describecluster to verify the nodes are in sync
- desc table from any node to verify column is added to the table
- select * from table limit 1 doesn't show the column
- insert into table (id, )values('some-id', alter table add column not always propogating
> -
>
> Key: CASSANDRA-7186
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7186
> Project: Cassandra
> Issue Type: Bug
>Reporter: Martin Meyer
>Assignee: Philip Thompson
> Fix For: 2.0.12
>
>
> I've been many times in Cassandra 2.0.6 that adding columns to existing
> tables seems to not fully propagate to our entire cluster. We add an extra
> column to various tables maybe 0-2 times a week, and so far many of these
> ALTERs have resulted in at least one node showing the old table description a
> pretty long time (~30 mins) after the original ALTER command was issued.
> We originally identified this issue when a connected clients would complain
> that a column it issued a SELECT for wasn't a known column, at which point we
> have to ask each node to describe the most recently altered table. One of
> them will not know about the newly added field. Issuing the original ALTER
> statement on that node makes everything work correctly.
> We have seen this issue on multiple tables (we don't always alter the same
> one). It has affected various nodes in the cluster (not always the same one
> is not getting the mutation propagated). No new nodes have been added to the
> cluster recently. All nodes are homogenous (hardware and software), running
> 2.0.6. We don't see any particular errors or exceptions on the node that
> didn't get the schema update, only the later error from a Java client about
> asking for an unknown column in a SELECT. We have to check each node manually
> to find the offender. The tables he have seen this on are under fairly heavy
> read and write load, but we haven't altered any tables that are not, so that
> might not be important.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
