Re: consistency level question
Thanks, *Tamar Fraenkel * Senior Software Engineer, TOK Media [image: Inline image 1] ta...@tok-media.com Tel: +972 2 6409736 Mob: +972 54 8356490 Fax: +972 2 5612956 On Mon, Mar 19, 2012 at 11:49 AM, aaron morton wrote: > Some information on node failures, consistency levels and availability > http://thelastpickle.com/2011/06/13/Down-For-Me/ > > Cheers > > > - > Aaron Morton > Freelance Developer > @aaronmorton > http://www.thelastpickle.com > > On 19/03/2012, at 1:08 PM, Watanabe Maki wrote: > > Yes, read and write won't fail with single node failure. > But your read may return old data. > > maki > > On 2012/03/19, at 1:08, Caleb Rackliffe wrote: > > That sounds right to me :) > > * > Caleb Rackliffe | Software Developer > M 949.981.0159 | ca...@steelhouse.com > ** > > * > > From: Tamar Fraenkel > Reply-To: "user@cassandra.apache.org" > Date: Sun, 18 Mar 2012 04:20:58 -0400 > To: "user@cassandra.apache.org" > Subject: Re: consistency level question > > Thanks! > I updated replication factor to 2, and now when I took one node down all > continued running (I did see Hector complaining on the node being down), > but things were saved to db and read from it. > > Just so I understand, now, having replication factor of 2, if I have 2 out > of 3 nodes running all my read and writes with CL=1 should work, right? > > > *Tamar Fraenkel * > Senior Software Engineer, TOK Media > > > > ta...@tok-media.com > Tel: +972 2 6409736 > Mob: +972 54 8356490 > Fax: +972 2 5612956 > > > > > > On Sun, Mar 18, 2012 at 9:57 AM, Watanabe Maki wrote: > >> Because your RF is 1, so you need all nodes up. >> >> maki >> >> >> On 2012/03/18, at 16:15, Tamar Fraenkel wrote: >> >> Hi! >> I have a 3 node cassandra cluster. >> I use Hector API. >> >> I give hecotr one of the node's IP address >> I call setAutoDiscoverHosts(true) and setRunAutoDiscoveryAtStartup(true). >> >> The describe on one node returns: >> >> Replication Strategy: org.apache.cassandra.locator.SimpleStrategy >> Durable Writes: true >> Options: [replication_factor:1] >> >> The odd thing is that when I take one of the nodes down, expecting all to >> continue running smoothly, I get exceptions of the format seen bellow, and >> no read or write succeeds. When I bring the node back up, exceptions stop >> and read and write resumes. >> >> Any idea or explanation why this is the case? >> Thanks! >> >> >> me.prettyprint.hector.api.exceptions.HUnavailableException: : May not be >> enough replicas present to handle consistency level. >> at >> me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:66) >> at >> me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:285) >> at >> me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:268) >> at >> me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:103) >> at >> me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:246) >> at >> me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(KeyspaceServiceImpl.java:131) >> at >> me.prettyprint.cassandra.service.KeyspaceServiceImpl.getSlice(KeyspaceServiceImpl.java:289) >> at >> me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:53) >> at >> me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:49) >> at >> me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMeasure(KeyspaceOperationCallback.java:20) >> at >> me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspace.java:85) >> at >> me.prettyprint.cassandra.model.thrift.ThriftSliceQuery.execute(ThriftSliceQuery.java:48) >> at >> me.prettyprint.cassandra.service.ColumnSliceIterator.hasNext(ColumnSliceIterator.java:60) >> at >> >> >> *Tamar Fraenkel * >> Senior Software Engineer, TOK Media >> >> >> >> >> ta...@tok-media.com >> Tel: +972 2 6409736 >> Mob: +972 54 8356490 >> Fax: +972 2 5612956 >> >> >> >> > > <>
Re: consistency level question
Some information on node failures, consistency levels and availability http://thelastpickle.com/2011/06/13/Down-For-Me/ Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 19/03/2012, at 1:08 PM, Watanabe Maki wrote: > Yes, read and write won't fail with single node failure. > But your read may return old data. > > maki > > On 2012/03/19, at 1:08, Caleb Rackliffe wrote: > >> That sounds right to me :) >> >> Caleb Rackliffe | Software Developer >> M 949.981.0159 | ca...@steelhouse.com >> >> >> From: Tamar Fraenkel >> Reply-To: "user@cassandra.apache.org" >> Date: Sun, 18 Mar 2012 04:20:58 -0400 >> To: "user@cassandra.apache.org" >> Subject: Re: consistency level question >> >> Thanks! >> I updated replication factor to 2, and now when I took one node down all >> continued running (I did see Hector complaining on the node being down), but >> things were saved to db and read from it. >> >> Just so I understand, now, having replication factor of 2, if I have 2 out >> of 3 nodes running all my read and writes with CL=1 should work, right? >> >> >> Tamar Fraenkel >> Senior Software Engineer, TOK Media >> >> >> >> ta...@tok-media.com >> Tel: +972 2 6409736 >> Mob: +972 54 8356490 >> Fax: +972 2 5612956 >> >> >> >> >> >> On Sun, Mar 18, 2012 at 9:57 AM, Watanabe Maki >> wrote: >> Because your RF is 1, so you need all nodes up. >> >> maki >> >> >> On 2012/03/18, at 16:15, Tamar Fraenkel wrote: >> >>> Hi! >>> I have a 3 node cassandra cluster. >>> I use Hector API. >>> >>> I give hecotr one of the node's IP address >>> I call setAutoDiscoverHosts(true) and setRunAutoDiscoveryAtStartup(true). >>> >>> The describe on one node returns: >>> >>> Replication Strategy: org.apache.cassandra.locator.SimpleStrategy >>> Durable Writes: true >>> Options: [replication_factor:1] >>> >>> The odd thing is that when I take one of the nodes down, expecting all to >>> continue running smoothly, I get exceptions of the format seen bellow, and >>> no read or write succeeds. When I bring the node back up, exceptions stop >>> and read and write resumes. >>> >>> Any idea or explanation why this is the case? >>> Thanks! >>> >>> >>> me.prettyprint.hector.api.exceptions.HUnavailableException: : May not be >>> enough replicas present to handle consistency level. >>> at >>> me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:66) >>> at >>> me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:285) >>> at >>> me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:268) >>> at >>> me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:103) >>> at >>> me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:246) >>> at >>> me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(KeyspaceServiceImpl.java:131) >>> at >>> me.prettyprint.cassandra.service.KeyspaceServiceImpl.getSlice(KeyspaceServiceImpl.java:289) >>> at >>> me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:53) >>> at >>> me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:49) >>> at >>> me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMeasure(KeyspaceOperationCallback.java:20) >>> at >>> me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspace.java:85) >>> at >>> me.prettyprint.cassandra.model.thrift.ThriftSliceQuery.execute(ThriftSliceQuery.java:48) >>> at >>> me.prettyprint.cassandra.service.ColumnSliceIterator.hasNext(ColumnSliceIterator.java:60) >>> at >>> >>> >>> Tamar Fraenkel >>> Senior Software Engineer, TOK Media >>> >>> >>> >>> >>> ta...@tok-media.com >>> Tel: +972 2 6409736 >>> Mob: +972 54 8356490 >>> Fax: +972 2 5612956 >>> >>> >>> >>
Re: consistency level question
Yes, read and write won't fail with single node failure. But your read may return old data. maki On 2012/03/19, at 1:08, Caleb Rackliffe wrote: > That sounds right to me :) > > Caleb Rackliffe | Software Developer > M 949.981.0159 | ca...@steelhouse.com > > > From: Tamar Fraenkel > Reply-To: "user@cassandra.apache.org" > Date: Sun, 18 Mar 2012 04:20:58 -0400 > To: "user@cassandra.apache.org" > Subject: Re: consistency level question > > Thanks! > I updated replication factor to 2, and now when I took one node down all > continued running (I did see Hector complaining on the node being down), but > things were saved to db and read from it. > > Just so I understand, now, having replication factor of 2, if I have 2 out of > 3 nodes running all my read and writes with CL=1 should work, right? > > > Tamar Fraenkel > Senior Software Engineer, TOK Media > > > > ta...@tok-media.com > Tel: +972 2 6409736 > Mob: +972 54 8356490 > Fax: +972 2 5612956 > > > > > > On Sun, Mar 18, 2012 at 9:57 AM, Watanabe Maki > wrote: > Because your RF is 1, so you need all nodes up. > > maki > > > On 2012/03/18, at 16:15, Tamar Fraenkel wrote: > >> Hi! >> I have a 3 node cassandra cluster. >> I use Hector API. >> >> I give hecotr one of the node's IP address >> I call setAutoDiscoverHosts(true) and setRunAutoDiscoveryAtStartup(true). >> >> The describe on one node returns: >> >> Replication Strategy: org.apache.cassandra.locator.SimpleStrategy >> Durable Writes: true >> Options: [replication_factor:1] >> >> The odd thing is that when I take one of the nodes down, expecting all to >> continue running smoothly, I get exceptions of the format seen bellow, and >> no read or write succeeds. When I bring the node back up, exceptions stop >> and read and write resumes. >> >> Any idea or explanation why this is the case? >> Thanks! >> >> >> me.prettyprint.hector.api.exceptions.HUnavailableException: : May not be >> enough replicas present to handle consistency level. >> at >> me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:66) >> at >> me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:285) >> at >> me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:268) >> at >> me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:103) >> at >> me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:246) >> at >> me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(KeyspaceServiceImpl.java:131) >> at >> me.prettyprint.cassandra.service.KeyspaceServiceImpl.getSlice(KeyspaceServiceImpl.java:289) >> at >> me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:53) >> at >> me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:49) >> at >> me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMeasure(KeyspaceOperationCallback.java:20) >> at >> me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspace.java:85) >> at >> me.prettyprint.cassandra.model.thrift.ThriftSliceQuery.execute(ThriftSliceQuery.java:48) >> at >> me.prettyprint.cassandra.service.ColumnSliceIterator.hasNext(ColumnSliceIterator.java:60) >> at >> >> >> Tamar Fraenkel >> Senior Software Engineer, TOK Media >> >> >> >> >> ta...@tok-media.com >> Tel: +972 2 6409736 >> Mob: +972 54 8356490 >> Fax: +972 2 5612956 >> >> >> >
Re: consistency level question
That sounds right to me :) Caleb Rackliffe | Software Developer M 949.981.0159 | ca...@steelhouse.com [cid:8E620335-844B-4EFF-ACAB-3D4439A3B4B6] From: Tamar Fraenkel mailto:ta...@tok-media.com>> Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" mailto:user@cassandra.apache.org>> Date: Sun, 18 Mar 2012 04:20:58 -0400 To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" mailto:user@cassandra.apache.org>> Subject: Re: consistency level question Thanks! I updated replication factor to 2, and now when I took one node down all continued running (I did see Hector complaining on the node being down), but things were saved to db and read from it. Just so I understand, now, having replication factor of 2, if I have 2 out of 3 nodes running all my read and writes with CL=1 should work, right? Tamar Fraenkel Senior Software Engineer, TOK Media [cid:ii_135b91fb888fa9ff] ta...@tok-media.com<mailto:ta...@tok-media.com> Tel: +972 2 6409736 Mob: +972 54 8356490 Fax: +972 2 5612956 On Sun, Mar 18, 2012 at 9:57 AM, Watanabe Maki mailto:watanabe.m...@gmail.com>> wrote: Because your RF is 1, so you need all nodes up. maki On 2012/03/18, at 16:15, Tamar Fraenkel mailto:ta...@tok-media.com>> wrote: Hi! I have a 3 node cassandra cluster. I use Hector API. I give hecotr one of the node's IP address I call setAutoDiscoverHosts(true) and setRunAutoDiscoveryAtStartup(true). The describe on one node returns: Replication Strategy: org.apache.cassandra.locator.SimpleStrategy Durable Writes: true Options: [replication_factor:1] The odd thing is that when I take one of the nodes down, expecting all to continue running smoothly, I get exceptions of the format seen bellow, and no read or write succeeds. When I bring the node back up, exceptions stop and read and write resumes. Any idea or explanation why this is the case? Thanks! me.prettyprint.hector.api.exceptions.HUnavailableException: : May not be enough replicas present to handle consistency level. at me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:66) at me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:285) at me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:268) at me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:103) at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:246) at me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(KeyspaceServiceImpl.java:131) at me.prettyprint.cassandra.service.KeyspaceServiceImpl.getSlice(KeyspaceServiceImpl.java:289) at me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:53) at me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:49) at me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMeasure(KeyspaceOperationCallback.java:20) at me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspace.java:85) at me.prettyprint.cassandra.model.thrift.ThriftSliceQuery.execute(ThriftSliceQuery.java:48) at me.prettyprint.cassandra.service.ColumnSliceIterator.hasNext(ColumnSliceIterator.java:60) at Tamar Fraenkel Senior Software Engineer, TOK Media ta...@tok-media.com<mailto:ta...@tok-media.com> Tel: +972 2 6409736 Mob: +972 54 8356490 Fax: +972 2 5612956 <><>
Re: consistency level question
Thanks! I updated replication factor to 2, and now when I took one node down all continued running (I did see Hector complaining on the node being down), but things were saved to db and read from it. Just so I understand, now, having replication factor of 2, if I have 2 out of 3 nodes running all my read and writes with CL=1 should work, right? *Tamar Fraenkel * Senior Software Engineer, TOK Media [image: Inline image 1] ta...@tok-media.com Tel: +972 2 6409736 Mob: +972 54 8356490 Fax: +972 2 5612956 On Sun, Mar 18, 2012 at 9:57 AM, Watanabe Maki wrote: > Because your RF is 1, so you need all nodes up. > > maki > > > On 2012/03/18, at 16:15, Tamar Fraenkel wrote: > > Hi! > I have a 3 node cassandra cluster. > I use Hector API. > > I give hecotr one of the node's IP address > I call setAutoDiscoverHosts(true) and setRunAutoDiscoveryAtStartup(true). > > The describe on one node returns: > > Replication Strategy: org.apache.cassandra.locator.SimpleStrategy > Durable Writes: true > Options: [replication_factor:1] > > The odd thing is that when I take one of the nodes down, expecting all to > continue running smoothly, I get exceptions of the format seen bellow, and > no read or write succeeds. When I bring the node back up, exceptions stop > and read and write resumes. > > Any idea or explanation why this is the case? > Thanks! > > > me.prettyprint.hector.api.exceptions.HUnavailableException: : May not be > enough replicas present to handle consistency level. > at > me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:66) > at > me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:285) > at > me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:268) > at > me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:103) > at > me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:246) > at > me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(KeyspaceServiceImpl.java:131) > at > me.prettyprint.cassandra.service.KeyspaceServiceImpl.getSlice(KeyspaceServiceImpl.java:289) > at > me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:53) > at > me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:49) > at > me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMeasure(KeyspaceOperationCallback.java:20) > at > me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspace.java:85) > at > me.prettyprint.cassandra.model.thrift.ThriftSliceQuery.execute(ThriftSliceQuery.java:48) > at > me.prettyprint.cassandra.service.ColumnSliceIterator.hasNext(ColumnSliceIterator.java:60) > at > > > *Tamar Fraenkel * > Senior Software Engineer, TOK Media > > > > > ta...@tok-media.com > Tel: +972 2 6409736 > Mob: +972 54 8356490 > Fax: +972 2 5612956 > > > > <>
Re: consistency level question
Because your RF is 1, so you need all nodes up. maki On 2012/03/18, at 16:15, Tamar Fraenkel wrote: > Hi! > I have a 3 node cassandra cluster. > I use Hector API. > > I give hecotr one of the node's IP address > I call setAutoDiscoverHosts(true) and setRunAutoDiscoveryAtStartup(true). > > The describe on one node returns: > > Replication Strategy: org.apache.cassandra.locator.SimpleStrategy > Durable Writes: true > Options: [replication_factor:1] > > The odd thing is that when I take one of the nodes down, expecting all to > continue running smoothly, I get exceptions of the format seen bellow, and no > read or write succeeds. When I bring the node back up, exceptions stop and > read and write resumes. > > Any idea or explanation why this is the case? > Thanks! > > > me.prettyprint.hector.api.exceptions.HUnavailableException: : May not be > enough replicas present to handle consistency level. > at > me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:66) > at > me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:285) > at > me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:268) > at > me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:103) > at > me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:246) > at > me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(KeyspaceServiceImpl.java:131) > at > me.prettyprint.cassandra.service.KeyspaceServiceImpl.getSlice(KeyspaceServiceImpl.java:289) > at > me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:53) > at > me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:49) > at > me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMeasure(KeyspaceOperationCallback.java:20) > at > me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspace.java:85) > at > me.prettyprint.cassandra.model.thrift.ThriftSliceQuery.execute(ThriftSliceQuery.java:48) > at > me.prettyprint.cassandra.service.ColumnSliceIterator.hasNext(ColumnSliceIterator.java:60) > at > > > Tamar Fraenkel > Senior Software Engineer, TOK Media > > > > ta...@tok-media.com > Tel: +972 2 6409736 > Mob: +972 54 8356490 > Fax: +972 2 5612956 > > >
Re: consistency level question
Hi! Thanks for the prompt answer, That is true, I intend to have it at two. What you say, is that if I change that, then even when the node is down, my application will be able to read\write from the other node where the data is replicated? Forgot to mention that I have ConfigurableConsistencyLevel ccl = new ConfigurableConsistencyLevel(); ccl.setDefaultReadConsistencyLevel(HConsistencyLevel.ONE); ccl.setDefaultWriteConsistencyLevel(HConsistencyLevel.ONE); keyspace = HFactory.createKeyspace(KEYSPACE, cluster, ccl, new FailoverPolicy(3, 10)); Thanks, *Tamar Fraenkel * Senior Software Engineer, TOK Media [image: Inline image 1] ta...@tok-media.com Tel: +972 2 6409736 Mob: +972 54 8356490 Fax: +972 2 5612956 On Sun, Mar 18, 2012 at 9:31 AM, Caleb Rackliffe wrote: > If your replication factor is set to one, your cluster is obviously in a > bad state following any node failure. At best, I think it would make sense > that about a third of your operations fail, but I'm not sure why all of > them would. I don't know if Hector just refuses to work with a compromised > cluster, etc. > > I guess I'm wondering why your replication factor is set to 1… > > * > Caleb Rackliffe | Software Developer > M 949.981.0159 | ca...@steelhouse.com > ** > * > > From: Tamar Fraenkel > Reply-To: "user@cassandra.apache.org" > Date: Sun, 18 Mar 2012 03:15:53 -0400 > To: "cassandra-u...@incubator.apache.org" < > cassandra-u...@incubator.apache.org> > Subject: consistency level question > > Hi! > I have a 3 node cassandra cluster. > I use Hector API. > > I give hecotr one of the node's IP address > I call setAutoDiscoverHosts(true) and setRunAutoDiscoveryAtStartup(true). > > The describe on one node returns: > > Replication Strategy: org.apache.cassandra.locator.SimpleStrategy > Durable Writes: true > Options: [replication_factor:1] > > The odd thing is that when I take one of the nodes down, expecting all to > continue running smoothly, I get exceptions of the format seen bellow, and > no read or write succeeds. When I bring the node back up, exceptions stop > and read and write resumes. > > Any idea or explanation why this is the case? > Thanks! > > > me.prettyprint.hector.api.exceptions.HUnavailableException: : May not be > enough replicas present to handle consistency level. > at > me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:66) > at > me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:285) > at > me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:268) > at > me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:103) > at > me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:246) > at > me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(KeyspaceServiceImpl.java:131) > at > me.prettyprint.cassandra.service.KeyspaceServiceImpl.getSlice(KeyspaceServiceImpl.java:289) > at > me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:53) > at > me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:49) > at > me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMeasure(KeyspaceOperationCallback.java:20) > at > me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspace.java:85) > at > me.prettyprint.cassandra.model.thrift.ThriftSliceQuery.execute(ThriftSliceQuery.java:48) > at > me.prettyprint.cassandra.service.ColumnSliceIterator.hasNext(ColumnSliceIterator.java:60) > at > > > *Tamar Fraenkel * > Senior Software Engineer, TOK Media > > [image: Inline image 1] > > ta...@tok-media.com > Tel: +972 2 6409736 > Mob: +972 54 8356490 > Fax: +972 2 5612956 > > > > <><><>
Re: consistency level question
If your replication factor is set to one, your cluster is obviously in a bad state following any node failure. At best, I think it would make sense that about a third of your operations fail, but I'm not sure why all of them would. I don't know if Hector just refuses to work with a compromised cluster, etc. I guess I'm wondering why your replication factor is set to 1… Caleb Rackliffe | Software Developer M 949.981.0159 | ca...@steelhouse.com [cid:3CCB142F-1DF5-423E-BF0E-4D8F3F31E15B] From: Tamar Fraenkel mailto:ta...@tok-media.com>> Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" mailto:user@cassandra.apache.org>> Date: Sun, 18 Mar 2012 03:15:53 -0400 To: "cassandra-u...@incubator.apache.org<mailto:cassandra-u...@incubator.apache.org>" mailto:cassandra-u...@incubator.apache.org>> Subject: consistency level question Hi! I have a 3 node cassandra cluster. I use Hector API. I give hecotr one of the node's IP address I call setAutoDiscoverHosts(true) and setRunAutoDiscoveryAtStartup(true). The describe on one node returns: Replication Strategy: org.apache.cassandra.locator.SimpleStrategy Durable Writes: true Options: [replication_factor:1] The odd thing is that when I take one of the nodes down, expecting all to continue running smoothly, I get exceptions of the format seen bellow, and no read or write succeeds. When I bring the node back up, exceptions stop and read and write resumes. Any idea or explanation why this is the case? Thanks! me.prettyprint.hector.api.exceptions.HUnavailableException: : May not be enough replicas present to handle consistency level. at me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:66) at me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:285) at me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:268) at me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:103) at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:246) at me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(KeyspaceServiceImpl.java:131) at me.prettyprint.cassandra.service.KeyspaceServiceImpl.getSlice(KeyspaceServiceImpl.java:289) at me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:53) at me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:49) at me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMeasure(KeyspaceOperationCallback.java:20) at me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspace.java:85) at me.prettyprint.cassandra.model.thrift.ThriftSliceQuery.execute(ThriftSliceQuery.java:48) at me.prettyprint.cassandra.service.ColumnSliceIterator.hasNext(ColumnSliceIterator.java:60) at Tamar Fraenkel Senior Software Engineer, TOK Media [cid:ii_135b91fb888fa9ff] ta...@tok-media.com<mailto:ta...@tok-media.com> Tel: +972 2 6409736 Mob: +972 54 8356490 Fax: +972 2 5612956 <><>
consistency level question
Hi! I have a 3 node cassandra cluster. I use Hector API. I give hecotr one of the node's IP address I call setAutoDiscoverHosts(true) and setRunAutoDiscoveryAtStartup(true). The describe on one node returns: Replication Strategy: org.apache.cassandra.locator.SimpleStrategy Durable Writes: true Options: [replication_factor:1] The odd thing is that when I take one of the nodes down, expecting all to continue running smoothly, I get exceptions of the format seen bellow, and no read or write succeeds. When I bring the node back up, exceptions stop and read and write resumes. Any idea or explanation why this is the case? Thanks! me.prettyprint.hector.api.exceptions.HUnavailableException: : May not be enough replicas present to handle consistency level. at me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:66) at me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:285) at me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:268) at me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:103) at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:246) at me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(KeyspaceServiceImpl.java:131) at me.prettyprint.cassandra.service.KeyspaceServiceImpl.getSlice(KeyspaceServiceImpl.java:289) at me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:53) at me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:49) at me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMeasure(KeyspaceOperationCallback.java:20) at me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspace.java:85) at me.prettyprint.cassandra.model.thrift.ThriftSliceQuery.execute(ThriftSliceQuery.java:48) at me.prettyprint.cassandra.service.ColumnSliceIterator.hasNext(ColumnSliceIterator.java:60) at *Tamar Fraenkel * Senior Software Engineer, TOK Media [image: Inline image 1] ta...@tok-media.com Tel: +972 2 6409736 Mob: +972 54 8356490 Fax: +972 2 5612956 <>
