incremental backups not completely working?
Today I configured incremental backups in a test node which already has some data on it, and I found that backups are not created for STTables created by a compact: mddione@life:~/src/works/orange/Cassandra$ sudo find /var/lib/cassandra/data/one_cf /var/lib/cassandra/data/one_cf /var/lib/cassandra/data/one_cf/cf_1 /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-4-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-4-Data.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-5-Index.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-4-Index.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-5-Filter.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-5-Data.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-4-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-5-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Index.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Data.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Filter.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Index.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Data.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Index.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Filter.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Filter.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Data.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Data.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Filter.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Index.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-4-Filter.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-5-Statistics.db mddione@life:~/src/works/orange/Cassandra$ nodetool compact mddione@life:~/src/works/orange/Cassandra$ sudo find /var/lib/cassandra/data/one_cf /var/lib/cassandra/data/one_cf /var/lib/cassandra/data/one_cf/cf_1 /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-6-Data.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-6-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-6-Index.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-6-Filter.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-6-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Index.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Data.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Filter.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Index.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Data.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Index.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Filter.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Filter.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Data.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Data.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Filter.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Index.db as you can see, neither SSTables 4 or 6 have backups; both were created by compactions. -- Marcos Dione SysAdmin Astek Sud-Est pour FT/TGPF/OPF/PORTAIL/DOP/HEBEX @ Marco Polo 04 97 12 62 45 - mdione@orange.com _ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre
new node joins the cluster but can't drop schemas
We used to have a nice test cluster with 2 nodes and everything was peachy. At some point we (re)added a third node, which seems to work allright. But then we try to delete one CF and requery it and we get this: root@pnscassandra03:~# cqlsh -3 [cqlsh 2.2.0 | Cassandra 1.1.2hebex1 | CQL spec 3.0.0 | Thrift protocol 19.32.0] cqlsh:test_restore drop table cf1; cqlsh:test_restore select count (*) from cf1; count --- 6703 Of course if we drop the CF from nodes 1 or 2 it disappears as expected: cqlsh:test_restore drop table cf1; cqlsh:test_restore select count (*) from cf1; Bad Request: unconfigured columnfamily cf1 As it's a test cluster and several people had used it on and off to do some tests, including the restore tests I posted about last week, I can't say exactly what happened with this machines. All I can say it's that logs look good[1]. I also have the debug logs if they're useful. Other info in the setup: Virtual machines running ubuntu 12.04, sun's java6, and nothing much changed from the default config. KS created as: CREATE KEYSPACE test_restore WITH strategy_class = 'SimpleStrategy' AND strategy_options:replication_factor = '2'; USE test_restore; CREATE TABLE cf2 ( id int PRIMARY KEY, val text ) WITH comment='' AND comparator=text AND read_repair_chance=0.10 AND gc_grace_seconds=864000 AND default_validation=text AND min_compaction_threshold=4 AND max_compaction_threshold=32 AND replicate_on_write='true' AND compaction_strategy_class='SizeTieredCompactionStrategy' AND compression_parameters:sstable_compression='SnappyCompressor'; and the deceased cf1 is similar to cf2. Any ideas? -- [1] http://pastebin.lugmen.org.ar/7647 -- Marcos Dione SysAdmin Astek Sud-Est pour FT/TGPF/OPF/PORTAIL/DOP/HEBEX @ Marco Polo 04 97 12 62 45 - mdione@orange.com _ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified. Thank you.
RE: new node joins the cluster but can't drop schemas
De : mdione@orange.com [mailto:mdione@orange.com] We used to have a nice test cluster with 2 nodes and everything was peachy. At some point we (re)added a third node, which seems to work allright. But then we try to delete one CF and requery it and we get this: Seems we've got biten by (at least one of) these bugs: https://issues.apache.org/jira/browse/CASSANDRA-4432 https://issues.apache.org/jira/browse/CASSANDRA-4472 Sorry for the noise. _ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified. Thank you.
RE: RE Restore snapshot
De : mdione@orange.com [mailto:mdione@orange.com] In particular, I'm thinking on a restore like this: * the app does something stupid. * (if possible) I stop writes to the KS or CF. In fact, given that I'm about to restore the KS/CF to an old state, I can safely do this: * drop and restore the schema. * remove the db files with the wrong data. * I put the db files from the restore. * nodetool refresh * profit! This works perfectly. _ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified. Thank you.
RE: RE Restore snapshot
De : Sylvain Lebresne [mailto:sylv...@datastax.com] 2) copy the snapshot sstable in the right place and call the JMX method loadNewSSTables() (in the column family MBean, which mean you need to do that per-CF). How does this affect the contents of the CommitLogs? I mean, I imagine using this for rollback'ing errors in the app side, like some/many/all objects deleted. How will updates/deletes saved in CommitLogs impact the data restored this way? Also: while I'm restoring a KS of CF, is it possible to cut write requests but not reads? I hope I'm explaining myself clearly... In particular, I'm thinking on a restore like this: * the app does something stupid. * (if possible) I stop writes to the KS or CF. * remove the db files with the wrong data. * I put the db files from the restore. * nodetool refresh * profit! But on Friday I did it slightly different (too tired, I was confused), and only did the refresh on one node and then a repair on the same, and it failed like this: * state just after the app did something stupid: root@pnscassandra04:~# tree /var/opt/hosting/db/cassandra/data/one_cf/cf_1/ /var/opt/hosting/db/cassandra/data/one_cf/cf_1/ |-- one_cf-cf_1-hd-13-CompressionInfo.db |-- one_cf-cf_1-hd-13-Data.db |-- one_cf-cf_1-hd-13-Filter.db |-- one_cf-cf_1-hd-13-Index.db |-- one_cf-cf_1-hd-13-Statistics.db |-- one_cf-cf_1-hd-14-CompressionInfo.db |-- one_cf-cf_1-hd-14-Data.db |-- one_cf-cf_1-hd-14-Filter.db |-- one_cf-cf_1-hd-14-Index.db |-- one_cf-cf_1-hd-14-Statistics.db |-- one_cf-cf_1-hd-15-CompressionInfo.db |-- one_cf-cf_1-hd-15-Data.db |-- one_cf-cf_1-hd-15-Filter.db |-- one_cf-cf_1-hd-15-Index.db |-- one_cf-cf_1-hd-15-Statistics.db `-- snapshots `-- 1344609231532 |-- one_cf-cf_1-hd-10-CompressionInfo.db |-- one_cf-cf_1-hd-10-Data.db |-- one_cf-cf_1-hd-10-Filter.db |-- one_cf-cf_1-hd-10-Index.db |-- one_cf-cf_1-hd-10-Statistics.db |-- one_cf-cf_1-hd-11-CompressionInfo.db |-- one_cf-cf_1-hd-11-Data.db |-- one_cf-cf_1-hd-11-Filter.db |-- one_cf-cf_1-hd-11-Index.db |-- one_cf-cf_1-hd-11-Statistics.db |-- one_cf-cf_1-hd-12-CompressionInfo.db |-- one_cf-cf_1-hd-12-Data.db |-- one_cf-cf_1-hd-12-Filter.db |-- one_cf-cf_1-hd-12-Index.db |-- one_cf-cf_1-hd-12-Statistics.db |-- one_cf-cf_1-hd-9-CompressionInfo.db |-- one_cf-cf_1-hd-9-Data.db |-- one_cf-cf_1-hd-9-Filter.db |-- one_cf-cf_1-hd-9-Index.db `-- one_cf-cf_1-hd-9-Statistics.db * I remove the 'wrong' databases root@pnscassandra04:~# rm -v /var/opt/hosting/db/cassandra/data/one_cf/cf_1/*.db removed `/var/opt/hosting/db/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-13-CompressionInfo.db' removed `/var/opt/hosting/db/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-13-Data.db' removed `/var/opt/hosting/db/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-13-Filter.db' removed `/var/opt/hosting/db/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-13-Index.db' removed `/var/opt/hosting/db/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-13-Statistics.db' removed `/var/opt/hosting/db/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-14-CompressionInfo.db' removed `/var/opt/hosting/db/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-14-Data.db' removed `/var/opt/hosting/db/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-14-Filter.db' removed `/var/opt/hosting/db/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-14-Index.db' removed `/var/opt/hosting/db/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-14-Statistics.db' removed `/var/opt/hosting/db/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-15-CompressionInfo.db' removed `/var/opt/hosting/db/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-15-Data.db' removed `/var/opt/hosting/db/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-15-Filter.db' removed `/var/opt/hosting/db/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-15-Index.db' removed `/var/opt/hosting/db/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-15-Statistics.db' * restore root@pnscassandra04:~# for i in /var/opt/hosting/db/cassandra/data/one_cf/cf_1/snapshots/1344609231532/*; do ln -v $i /var/opt/hosting/db/cassandra/data/one_cf/cf_1/; done `/var/opt/hosting/db/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-10-CompressionInfo.db' = `/var/opt/hosting/db/cassandra/data/one_cf/cf_1/snapshots/1344609231532/one_cf-cf_1-hd-10-CompressionInfo.db' `/var/opt/hosting/db/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-10-Data.db' = `/var/opt/hosting/db/cassandra/data/one_cf/cf_1/snapshots/1344609231532/one_cf-cf_1-hd-10-Data.db' `/var/opt/hosting/db/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-10-Filter.db' = `/var/opt/hosting/db/cassandra/data/one_cf/cf_1/snapshots/1344609231532/one_cf-cf_1-hd-10-Filter.db' `/var/opt/hosting/db/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-10-Index.db' = `/var/opt/hosting/db/cassandra/data/one_cf/cf_1/snapshots/1344609231532/one_cf-cf_1-hd-10-Index.db'
RE: RE Restore snapshot
De : mdione@orange.com [mailto:mdione@orange.com] De : Sylvain Lebresne [mailto:sylv...@datastax.com] 2) copy the snapshot sstable in the right place and call the JMX method loadNewSSTables() (in the column family MBean, which mean you need to do that per-CF). How does this affect the contents of the CommitLogs? I mean, I imagine using this for rollback'ing errors in the app side, like some/many/all objects deleted. How will updates/deletes saved in CommitLogs impact the data restored this way? Also: while I'm restoring a KS of CF, is it possible to cut write requests but not reads? I hope I'm explaining myself clearly... In particular, I'm thinking on a restore like this: * the app does something stupid. * (if possible) I stop writes to the KS or CF. * remove the db files with the wrong data. * I put the db files from the restore. * nodetool refresh * profit! [...] * and here's the error at repair time: root@pnscassandra04:~# nodetool repair one_cf cf_1 INFO [RMI TCP Connection(56)-10.234.169.244] 2012-08-10 17:03:12,354 StorageService.java (line 1925) Starting repair command #3, repairing 5 ranges. INFO [AntiEntropySessions:5] 2012-08-10 17:03:12,358 AntiEntropyService.java (line 666) [repair #811be330-e2fc-11e1-- b64817724cbd] new session: will sync pnscassandra04.vdev.bas.s1.p.fti.net/10.234.169.244, /10.234.169.245, /10.234.169.241, /10.234.169.242, /10.234.169.243 on range (34028236692093846346337460743176821145,6805647338418769269267492148635 3642291] for one_cf.[cf_1] INFO [AntiEntropySessions:5] 2012-08-10 17:03:12,359 AntiEntropyService.java (line 871) [repair #811be330-e2fc- 11e1--b64817724cbd] requesting merkle trees for cf_1 (to [/10.234.169.245, /10.234.169.241, /10.234.169.242, /10.234.169.243, pnscassandra04.vdev.bas.s1.p.fti.net/10.234.169.244]) ERROR [ValidationExecutor:2] 2012-08-10 17:03:12,361 AbstractCassandraDaemon.java (line 134) Exception in thread Thread[ValidationExecutor:2,1,main] java.io.IOError: java.io.FileNotFoundException: /var/opt/hosting/db/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-13- Data.db (No such file or directory) [...] I understand that this procedure (refresh+repair in one node) makes absolutely no sense, but the error leads me to think that after refresh, the 'wrong' databases are still taken in account. Is it that the whole procedure is bad from the start and I should use SSTableLoader instead? Ok, I just did the refresh without the repair on all the nodes at the same time, and the error is still there. Should I drop the CF or KS before doing this kind of restore? -- Marcos Dione SysAdmin Astek Sud-Est pour FT/TGPF/OPF/PORTAIL/DOP/HEBEX @ Marco Polo 04 97 12 62 45 - mdione@orange.com _ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified. Thank you.
RE: RE Restore snapshot
De : Sylvain Lebresne [mailto:sylv...@datastax.com] 2) copy the snapshot sstable in the right place and call the JMX method loadNewSSTables() (in the column family MBean, which mean you need to do that per-CF). How does this affect the contents of the CommitLogs? I mean, I imagine using this for rollback'ing errors in the app side, like some/many/all objects deleted. How will updates/deletes saved in CommitLogs impact the data restored this way? Also: while I'm restoring a KS of CF, is it possible to cut write requests but not reads? I hope I'm explaining myself clearly... -- Marcos Dione SysAdmin Astek Sud-Est pour FT/TGPF/OPF/PORTAIL/DOP/HEBEX @ Marco Polo 04 97 12 62 45 - mdione@orange.com _ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified. Thank you.
restoring a counter
According to this post[1], one's supposed to start C* with -Dcassandra.renew_counter_id=true as one of the steps of restoring a counter column family. I have two questions related to this: a) how does that setting affect C* in a non-restoring start? b) if it's bad (for some value of that), should I stop C*+remove the setting+start C* after the value has been repaired? c) bonus question: wouldn't it be nice to change the init.d script to be able to add this kind of one-time settings? -- [1] http://www.datastax.com/dev/blog/whats-new-in-cassandra-0-8-part-2-counters -- Marcos Dione SysAdmin Astek Sud-Est pour FT/TGPF/OPF/PORTAIL/DOP/HEBEX @ Marco Polo 04 97 12 62 45 - mdione@orange.com _ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified. Thank you.
RE: restoring a counter
De : mdione@orange.com [mailto:mdione@orange.com] restoring a counter column family. I have two questions related to this: a) how does that setting affect C* in a non-restoring start? b) if it's bad (for some value of that), should I stop C*+remove the setting+start C* after the value has been repaired? c) bonus question: wouldn't it be nice to change the init.d script to be able to add this kind of one-time settings? And a fourth one: d) how would you restore from a full cluster crash? Assuming that I have snapshots done at ~the same time. _ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified. Thank you.
going back in time
One of the scenarios I have to have in account for a small Cassandra cluster (N=4) is restoring the data back in time. I will have full backups for 15 days, and it's possible that I will need to restore, let's say, the data from 10 days ago (don't ask, I'm not going into the details why). I know/suspect that restores by KeySpace/ColumnFamily are possible (we haven't tested yet), but I wonder if it would there any side effects of stopping all the nodes (assuming cut to the other KS are OK), restoring the SSTables, and starting the nodes one by one. So far we're thinking in RF=4, but also in RF=3 or at least RFN in the future. Is all this crazy talk or is it possible? Any side effects in the KS system and/or indexes to have in account? -- Marcos Dione SysAdmin Astek Sud-Est pour FT/TGPF/OPF/PORTAIL/DOP/HEBEX @ Marco Polo 04 97 12 62 45 - mdione@orange.com _ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified. Thank you.
RE: going back in time
De : Pierre-Yves Ritschard [mailto:p...@spootnik.org] Snapshot and restores are great for point in time recovery. There's no particular side-effect if you're willing to accept the downtime. Are you sure? The system KS has no book-keeping about the KSs/CFs? For instance, schema changes, etc? If you don't want to take your whole cluster offline you can use sstableloader as well. Sounds wonderful. _ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified. Thank you.
RE: Bad Request: No indexed columns present in by-columns clause with equals operator
De : aaron morton [mailto:aa...@thelastpickle.com] The secondary index will be build using the compaction features, you can check the progress with nodetool compactionstats When they are build the output from describe. will list the build indexes. Built indexes: [] Should I understand that when the indexes are finished being built a) the «Built indexes» list should be empty and b) there should be no pending compactions? Because that's exactly what I have now but I still can't use the column HBX_FIL_STATUS in the where clause... -- Marcos Dione SysAdmin Astek Sud-Est pour FT/TGPF/OPF/PORTAIL/DOP/HEBEX @ Marco Polo 04 97 12 62 45 - mdione@orange.com _ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified. Thank you.
RE: Bad Request: No indexed columns present in by-columns clause with equals operator
De : mdione@orange.com [mailto:mdione@orange.com] Should I understand that when the indexes are finished being built a) the «Built indexes» list should be empty and b) there should be no pending compactions? Because that's exactly what I have now but I still can't use the column HBX_FIL_STATUS in the where clause... Ok, answering myself a little: it's more strange than that. When indexes are finished being built, they appear in the list, and there should be no compactions or anything else in «nodetool compactionstats», or at least none related to indexes. Also useful: you'll see this type of messages in the system log (if the log level is low enough): INFO [Creating index: hbx_file_test2.hbx_file_test2_hbx_fil_date_idx] 2012-04-25 16:04:48,666 SecondaryIndex.java (line 191) Index build of hbx_file_test2.hbx_file_test2_hbx_fil_date_idx complete Now, back to my problem. We have a 6 node cluster in 2 datacenters, the KS has a CF of 4 (2 in each DC). I created the index in node 1, but node 4 did all the work (we have log level down to debug, so we can see these kind of things, and also the creation appears in the output of «nodetool compactionstats» as noted above). We have seen similar messages in 4 nodes, but not all 6. And now we can make the query on any of the 4 nodes with success. But, and of course there's a but, we can't in the other 2, we still get the error in the subject. And of course we see a difference in «describe avatars»: we get «Built indexes: [HBX_FILE.HBX_FILE_HBX_FIL_STATUS_idx]» for the 4 nodes where the query works, and «Built indexes: []». No surprises there, except that it doesn't make any sense to us. How to continue? -- Marcos Dione SysAdmin Astek Sud-Est pour FT/TGPF/OPF/PORTAIL/DOP/HEBEX @ Marco Polo 04 97 12 62 45 - mdione@orange.com _ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified. Thank you.
RE: Bad Request: No indexed columns present in by-columns clause with equals operator
De : mdione@orange.com [mailto:mdione@orange.com] [default@avatars] describe HBX_FILE; ColumnFamily: HBX_FILE Key Validation Class: org.apache.cassandra.db.marshal.BytesType Default column value validator: org.apache.cassandra.db.marshal.BytesType Columns sorted by: org.apache.cassandra.db.marshal.UTF8Type Row cache size / save period in seconds / keys to save : 0.0/0/all Row Cache Provider: org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider Key cache size / save period in seconds: 20.0/14400 GC grace seconds: 864000 Compaction min/max thresholds: 4/32 Read repair chance: 1.0 Replicate on write: true Bloom Filter FP chance: default Built indexes: [] Column Metadata: Column Name: HBX_FIL_DATE Validation Class: org.apache.cassandra.db.marshal.UTF8Type Column Name: HBX_FIL_LARGE Validation Class: org.apache.cassandra.db.marshal.AsciiType Column Name: HBX_FIL_MEDIUM Validation Class: org.apache.cassandra.db.marshal.AsciiType Column Name: HBX_FIL_SMALL Validation Class: org.apache.cassandra.db.marshal.AsciiType Column Name: HBX_FIL_STATUS Validation Class: org.apache.cassandra.db.marshal.UTF8Type Index Name: HBX_FILE_HBX_FIL_STATUS_idx Index Type: KEYS Column Name: HBX_FIL_TINY Validation Class: org.apache.cassandra.db.marshal.AsciiType Compaction Strategy: org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy Someone in #cassandra pointed out that the index might be created, but it's shown as not built («Built indexes: []»). Is that right? Any idea how to build it? -- Marcos Dione SysAdmin Astek Sud-Est pour FT/TGPF/OPF/PORTAIL/DOP/HEBEX @ Marco Polo 04 97 12 62 45 - mdione@orange.com _ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified. Thank you.
Bad Request: No indexed columns present in by-columns clause with equals operator
I understand the error message, but I don't understand why I get it. Here's the CF: cqlsh:avatars describe columnfamily HBX_FILE; CREATE COLUMNFAMILY HBX_FILE ( KEY blob PRIMARY KEY, HBX_FIL_DATE text, HBX_FIL_LARGE ascii, HBX_FIL_MEDIUM ascii, HBX_FIL_SMALL ascii, HBX_FIL_STATUS text, HBX_FIL_TINY ascii ) WITH comment='' AND comparator=text AND read_repair_chance=1.00 AND gc_grace_seconds=864000 AND default_validation=blob AND min_compaction_threshold=4 AND max_compaction_threshold=32 AND replicate_on_write=True; CREATE INDEX HBX_FILE_HBX_FIL_STATUS_idx ON HBX_FILE (HBX_FIL_STATUS); The query and the error: cqlsh:avatars SELECT HBX_FIL_SMALL FROM HBX_FILE WHERE KEY=1 AND HBX_FIL_STATUS='actif'; Bad Request: No indexed columns present in by-columns clause with equals operator A query that works: cqlsh:avatars SELECT HBX_FIL_STATUS FROM HBX_FILE WHERE KEY=1; HBX_FIL_STATUS Actif Just in case, here's cli's output for the same CF: [default@avatars] describe HBX_FILE; ColumnFamily: HBX_FILE Key Validation Class: org.apache.cassandra.db.marshal.BytesType Default column value validator: org.apache.cassandra.db.marshal.BytesType Columns sorted by: org.apache.cassandra.db.marshal.UTF8Type Row cache size / save period in seconds / keys to save : 0.0/0/all Row Cache Provider: org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider Key cache size / save period in seconds: 20.0/14400 GC grace seconds: 864000 Compaction min/max thresholds: 4/32 Read repair chance: 1.0 Replicate on write: true Bloom Filter FP chance: default Built indexes: [] Column Metadata: Column Name: HBX_FIL_DATE Validation Class: org.apache.cassandra.db.marshal.UTF8Type Column Name: HBX_FIL_LARGE Validation Class: org.apache.cassandra.db.marshal.AsciiType Column Name: HBX_FIL_MEDIUM Validation Class: org.apache.cassandra.db.marshal.AsciiType Column Name: HBX_FIL_SMALL Validation Class: org.apache.cassandra.db.marshal.AsciiType Column Name: HBX_FIL_STATUS Validation Class: org.apache.cassandra.db.marshal.UTF8Type Index Name: HBX_FILE_HBX_FIL_STATUS_idx Index Type: KEYS Column Name: HBX_FIL_TINY Validation Class: org.apache.cassandra.db.marshal.AsciiType Compaction Strategy: org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy And the same error, with other words, in the CLI: [default@avatars] get HBX_FILE where HBX_FIL_STATUS = 'actif'; No indexed columns present in index clause with operator EQ Am I missing something? Might as well be that I'm too tired... -- Marcos Dione SysAdmin Astek Sud-Est pour FT/TGPF/OPF/PORTAIL/DOP/HEBEX @ Marco Polo 04 97 12 62 45 - mdione@orange.com _ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified. Thank you.
RE: Bad Request: No indexed columns present in by-columns clause with equals operator
De : Dave Brosius [mailto:dbros...@mebigfatguy.com] Works for me on trunk... what version are you using? Beh, I forgot that detail: 1.0.9. -- Marcos Dione SysAdmin Astek Sud-Est pour FT/TGPF/OPF/PORTAIL/DOP/HEBEX @ Marco Polo 04 97 12 62 45 - mdione@orange.com _ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified. Thank you.
RE: Bad Request: No indexed columns present in by-columns clause with equals operator
De : mdione@orange.com [mailto:mdione@orange.com] De : Dave Brosius [mailto:dbros...@mebigfatguy.com] Works for me on trunk... what version are you using? Beh, I forgot that detail: 1.0.9. I also forgot to mention: the index was recently created, after the database was populated, but there is not much data in the database. -- Marcos Dione SysAdmin Astek Sud-Est pour FT/TGPF/OPF/PORTAIL/DOP/HEBEX @ Marco Polo 04 97 12 62 45 - mdione@orange.com _ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified. Thank you.
RE: blob fields, bynary or hexa?
De : phuduc nguyen [mailto:duc.ngu...@pearson.com] How are you passing a blob or binary stream to the CLI? It sounds like you're passing in a representation of a binary stream as ascii/UTF8 which will create the problems you describe. So this is only a limitation of Cassandra-cli? -- Marcos Dione SysAdmin Astek Sud-Est pour FT/TGPF/OPF/PORTAIL/DOP/HEBEX @ Marco Polo 04 97 12 62 45 - mdione@orange.com _ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified. Thank you.
blob fields, bynary or hexa?
We're building a database to stock the avatars for our users in three sizes. Thing is, We planned to use the blob field with a ByteType validator, but if we try to inject the binary data as read from the image file, we get a cannot parse as hex bytes error. The same happens if we convert the binary data to its base64 representation. So far the only solutions we found is to actually convert the binary data to its 'string of hexa representations of each byte', meaning that each binary byte is actually stocked as two 'ascii bytes'; or, on the other hand, convert to bas64 and store it in a ascii column. Did we miss something? -- Marcos Dione SysAdmin Astek Sud-Est pour FT/TGPF/OPF/PORTAIL/DOP/HEBEX @ Marco Polo 04 97 12 62 45 - mdione@orange.com _ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified. Thank you.
RE: blob fields, bynary or hexa?
De : Erik Forkalsud [mailto:eforkals...@cj.com] Which client are you using? With Hector or straight thrift, your should be able to store byte[] directly. So far, cassandra-cli only, but we're also testing phpcassa with CQL support[1]. -- [1] https://github.com/thobbs/phpcassa -- Marcos Dione SysAdmin Astek Sud-Est pour FT/TGPF/OPF/PORTAIL/DOP/HEBEX @ Marco Polo 04 97 12 62 45 - mdione@orange.com _ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified. Thank you.
