RE: Interesting use case

2016-06-08 Thread Peer, Oded
Why do you think the amount of partitions is different in these tables? The partition key is the same (system_name and event_name). The number of rows per partition is different. From: kurt Greaves [mailto:k...@instaclustr.com] Sent: Thursday, June 09, 2016 7:52 AM To:

RE: Mapping a continuous range to a discrete value

2016-04-10 Thread Peer, Oded
s replacing. My new solution goes back to my old solution. On Thu, Apr 7, 2016 at 1:37 AM, Peer, Oded <oded.p...@rsa.com<mailto:oded.p...@rsa.com>> wrote: I have a table mapping continuous ranges to discrete values. CREATE TABLE range_mapping (k int, lower int, upper int, mapped_

Mapping a continuous range to a discrete value

2016-04-07 Thread Peer, Oded
I have a table mapping continuous ranges to discrete values. CREATE TABLE range_mapping (k int, lower int, upper int, mapped_value int, PRIMARY KEY (k, lower, upper)); INSERT INTO range_mapping (k, lower, upper, mapped_value) VALUES (0, 0, 99, 0); INSERT INTO range_mapping (k, lower, upper,

RE: Large number of tombstones without delete or update

2016-03-24 Thread Peer, Oded
ould I “unset” a field inside the JSON message written to the event_by_patient_timestamp table? Ralf On 24.03.2016, at 10:22, Peer, Oded <oded.p...@rsa.com<mailto:oded.p...@rsa.com>> wrote: http://www.datastax.com/dev/blog/datastax-java-driver-3-0-0-released#unset-values “For Protoco

RE: Large number of tombstones without delete or update

2016-03-24 Thread Peer, Oded
http://www.datastax.com/dev/blog/datastax-java-driver-3-0-0-released#unset-values "For Protocol V3 or below, all variables in a statement must be bound. With Protocol V4, variables can be left "unset", in which case they will be ignored server-side (no tombstones will be generated)." From:

RE: DataModelling to query date range

2016-03-24 Thread Peer, Oded
You can change the table to support Multi-column slice restrictions CREATE TABLE routes ( start text, end text, year int, month int, day int, PRIMARY KEY (start, end, year, month, day) ); Then using Multi-column slice restrictions you can query: SELECT * from routes where start = 'New York' and

Restoring a snapshot into a new cluster - thoughts on replica placement

2015-12-02 Thread Peer, Oded
I read the documentation for restoring a snapshot into a new cluster. It got me thinking about replica placement in that context. "NetworkTopologyStrategy places replicas in the same data center by walking the ring clockwise until reaching the first node in another rack." It seems it is not

RE: No query results while expecting results

2015-11-24 Thread Peer, Oded
Ramon, Have you tried another driver to determine if the problem is in the Python driver? You can deserialize your composite key using the following code: ByteBuffer t = ByteBufferUtil.hexToBytes("0008000e70451f6404000500"); short periodlen =

RE: No query results while expecting results

2015-11-23 Thread Peer, Oded
It might be a consistency issue. Assume your data for tnt 5 should be on nodes 1 and 2, but actually never got to node 1 for various reasons, and the hint wasn’t replayed for some reason and you didn’t run repairs. The data for tnt 5 is only on node 2. A query without restrictions on the

RE: Strategy tools for taking snapshots to load in another cluster instance

2015-11-19 Thread Peer, Oded
Have you read the DataStax documentation? http://docs.datastax.com/en/cassandra/2.0/cassandra/operations/ops_snapshot_restore_new_cluster.html From: Romain Hardouin [mailto:romainh...@yahoo.fr] Sent: Wednesday, November 18, 2015 3:59 PM To: user@cassandra.apache.org Subject: Re: Strategy tools

RE: PrepareStatement BUG

2015-08-26 Thread Peer, Oded
See https://issues.apache.org/jira/browse/CASSANDRA-7910 From: joseph gao [mailto:gaojf.bok...@gmail.com] Sent: Wednesday, August 26, 2015 6:15 AM To: user@cassandra.apache.org Subject: Re: PrepareStatement BUG Hi, anybody knows how to resolve this problem? 2015-08-23 1:35 GMT+08:00 joseph gao

RE: Can't connect to Cassandra server

2015-07-27 Thread Peer, Oded
HEAP_NEWSIZE=1600M But I am still unable to start the server properly. But this time system.log has bit different logs. https://gist.github.com/cdwijayarathna/75f65a34d9e71829adaa Any idea on how to proceed? Thanks On Wed, Jul 22, 2015 at 11:54 AM, Peer, Oded oded.p...@rsa.commailto:oded.p

RE: Can't connect to Cassandra server

2015-07-22 Thread Peer, Oded
Setting system_memory_in_mb to 16 GB means the Cassandra heap size you are using is 4 GB. If you meant to use a 16GB heap you should uncomment the line #MAX_HEAP_SIZE=4G And set MAX_HEAP_SIZE=16G You should uncomment the HEAP_NEWSIZE setting as well. I would leave it with the default setting

RE: howto do sql query like in a relational database

2015-07-21 Thread Peer, Oded
Cassandra is a highly scalable, eventually consistent, distributed, structured key-value store http://wiki.apache.org/cassandra/ It is intended for searching by key. It has more querying options but it really shines when querying by key. Not all databases offer the same functionality. Both a

RE: Can't connect to Cassandra server

2015-07-20 Thread Peer, Oded
...@gmail.commailto:cdwijayarat...@gmail.com wrote: Hi Peer, https://gist.githubusercontent.com/cdwijayarathna/a14586a9e39a943f89a0/raw/system%20log This is the log of the last time I started the server, I couldn't found any error there. Thanks On Sun, Jul 19, 2015 at 5:56 PM, Peer, Oded oded.p

RE: Can't connect to Cassandra server

2015-07-19 Thread Peer, Oded
Are you sure your node is up? Do you get a result when running “nodetool –h 192.248.15.219 status”? From: Chamila Wijayarathna [mailto:cdwijayarat...@gmail.com] Sent: Sunday, July 19, 2015 1:53 PM To: user@cassandra.apache.org Subject: Re: Can't connect to Cassandra server Hi Umang, Tried your

RE: Can't connect to Cassandra server

2015-07-19 Thread Peer, Oded
$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) What is the reason for this? How can I fix this? Thank You! On Sun, Jul 19, 2015 at 4:52 PM, Peer, Oded oded.p...@rsa.commailto:oded.p...@rsa.com wrote: Are you sure your node is up? Do you get a result

RE: Example Data Modelling

2015-07-07 Thread Peer, Oded
The data model suggested isn’t optimal for the “end of month” query you want to run since you are not querying by partition key. The query would look like “select EmpID, FN, LN, basic from salaries where month = 1” which requires filtering and has unpredictable performance. For this type of

RE: PrepareStatement problem

2015-06-16 Thread Peer, Oded
I get the alter table message? How can I get and deal that message? 2015-06-15 16:45 GMT+08:00 Peer, Oded oded.p...@rsa.commailto:oded.p...@rsa.com: This only applies to “select *” queries where you don’t specify the column names. There is a reported bug and fixed in 2.1.3. See https

RE: PrepareStatement problem

2015-06-15 Thread Peer, Oded
This only applies to “select *” queries where you don’t specify the column names. There is a reported bug and fixed in 2.1.3. See https://issues.apache.org/jira/browse/CASSANDRA-7910 From: joseph gao [mailto:gaojf.bok...@gmail.com] Sent: Monday, June 15, 2015 10:52 AM To:

RE: Insert Vs Updates - Both create tombstones

2015-05-14 Thread Peer, Oded
13, 2015 at 3:45 PM, Peer, Oded oded.p...@rsa.commailto:oded.p...@rsa.com wrote: Under the assumption that when you update the columns you also update the TTL for the columns then a tombstone won’t be created for those columns. Remember that TTL is set on columns (or “cells”), not on rows, so

RE: Updating only modified records (where lastModified current date)

2015-05-13 Thread Peer, Oded
You can use the “last modified” value as the TIMESTAMP for your UPDATE operation. This way the values will only be updated if lastModified date the lastModified you have in the DB. Updates to values don’t create tombstones. Only deletes (either by executing delete, inserting a null value or

RE: Insert Vs Updates - Both create tombstones

2015-05-13 Thread Peer, Oded
Under the assumption that when you update the columns you also update the TTL for the columns then a tombstone won't be created for those columns. Remember that TTL is set on columns (or cells), not on rows, so your description of updating a row is slightly misleading. If every query updates

RE: Updating only modified records (where lastModified current date)

2015-05-13 Thread Peer, Oded
are present in queries), then is there no cost to just submitting an update for everything regardless of whether lastModified has changed? Thanks. On Wed, May 13, 2015 at 3:38 PM, Peer, Oded oded.p...@rsa.commailto:oded.p...@rsa.com wrote: You can use the “last modified” value as the TIMESTAMP

RE: Updating only modified records (where lastModified current date)

2015-05-13 Thread Peer, Oded
unless the timestamp is actually last update timestamp?) Also, is there a way to get the number of rows which were updated / ignored? On Wed, May 13, 2015 at 4:37 PM, Peer, Oded oded.p...@rsa.commailto:oded.p...@rsa.com wrote: The cost of issuing an UPDATE that won’t update anything is compaction

RE: Updating only modified records (where lastModified current date)

2015-05-13 Thread Peer, Oded
, or will they be ignored? E.g if I issued an update where TIMESTAMP is X, then 1 hour later I issued another update where TIMESTAMP is still X, will that 2nd update essentially get ignored, or will it cause any overhead? On Wed, May 13, 2015 at 5:02 PM, Peer, Oded oded.p...@rsa.commailto:oded.p...@rsa.com wrote

RE: Inserting null values

2015-05-07 Thread Peer, Oded
I’ve added an option to prevent tombstone creation when using PreparedStatements to trunk, see CASSANDRA-7304. The problem is having tombstones in regular columns. When you perform a read request (range query or by PK): - Cassandra iterates over all the cells (all, not only the cells specified

RE: Data Modeling for 2.1 Cassandra

2015-04-30 Thread Peer, Oded
In general your data model should match your queries in Cassandra. In the examples you provided the queries are by name, not by ID, so I don’t see much use in using ID as the primary key. Without much context, like why you are using SET or if queries must specify both first_name and last_name

RE: Inserting null values

2015-04-29 Thread Peer, Oded
Inserting a null value creates a tombstone. Tombstones can have major performance implications. You can see the tombstones using sstable2json. If you have a small number of records with null values this seems OK, otherwise I recommend using the QueryBuilder (for Java clients) and waiting for

RE: Tables showing up as our_table-147a2090ed4211e480153bc81e542ebd/ in data dir

2015-04-29 Thread Peer, Oded
See https://github.com/apache/cassandra/blob/trunk/NEWS.txt#L173 SSTable data directory name is slightly changed. Each directory will have hex string appended after CF name, e.g. ks/cf-5be396077b811e3a3ab9dc4b9ac088d/ This hex string part represents unique ColumnFamily ID. Note that existing

RE: Data model suggestions

2015-04-27 Thread Peer, Oded
). Thanks. On Sun, Apr 26, 2015 at 1:29 PM, Peer, Oded oded.p...@rsa.commailto:oded.p...@rsa.com wrote: I would maintain two tables. An “archive” table that holds all the active and inactive records, and is updated hourly (re-inserting the same record has some compaction overhead but on the other

RE: Data model suggestions

2015-04-26 Thread Peer, Oded
I would maintain two tables. An “archive” table that holds all the active and inactive records, and is updated hourly (re-inserting the same record has some compaction overhead but on the other side deleting records has tombstones overhead). An “active” table which holds all the records in the

RE: sstable writer and creating bytebuffers

2015-03-31 Thread Peer, Oded
On Tue, Mar 31, 2015 at 7:42 AM, Peer, Oded oded.p...@rsa.commailto:oded.p...@rsa.com wrote: Thanks Sylvain. Is there any way to create a composite key with only one column in Cassandra when creating a table, or should creating a CompositeType instance with a single column be prohibited? It's

sstable writer and creating bytebuffers

2015-03-30 Thread Peer, Oded
I am writing code to bulk load data into Cassandra using SSTableSimpleUnsortedWriter I changed my partition key from a composite key (long, int) to a single column key (long). For creating the composite key I used a CompositeType, and I kept using it after changing the key to a single column.

RE: sstable writer and creating bytebuffers

2015-03-30 Thread Peer, Oded
class javadoc if you're interested. It's not the most compact format there is but changing it would break backward compatibility anyway. On Mon, Mar 30, 2015 at 12:38 PM, Peer, Oded oded.p...@rsa.commailto:oded.p...@rsa.com wrote: I am writing code to bulk load data into Cassandra using

RE: Saving a file using cassandra

2015-03-30 Thread Peer, Oded
Try this http://stackoverflow.com/a/17208343/248656 From: jean paul [mailto:researche...@gmail.com] Sent: Wednesday, March 18, 2015 7:06 PM To: user@cassandra.apache.org Subject: Saving a file using cassandra Hello, Finally, i have created my ring using cassandra. Please, i'd like to store a

Configuring all nodes as seeds

2014-06-18 Thread Peer, Oded
My intended Cassandra cluster will have 15 nodes per DC, with 2 DCs. I am considering using all the nodes as seed nodes. It looks like having all the nodes as seeds should actually reduce the Gossip overhead (See Gossiper implementation in http://wiki.apache.org/cassandra/ArchitectureGossip) Is

RE: Problem using sstableloader with SSTableSimpleUnsortedWriter and a composite key

2013-06-30 Thread Peer, Oded
Thank you Aaaron! Your blog post helped me understand how a row with a compound key is stored and this helped me understand how to create the sstable files. For anyone who needs it this is how it works: In Cassandra-cli the row looks like this: RowKey: 5 = (column=10:created,

Problem using sstableloader with SSTableSimpleUnsortedWriter and a composite key

2013-06-27 Thread Peer, Oded
Hi, I am using Cassandra 1.2.5. I built a cluster of 2 data centers with 3 nodes in each data center. I created a key space and table with a composite key: create keyspace test_keyspace WITH replication = {'class': 'NetworkTopologyStrategy', 'DC1' : 1, 'DC2' : 1}; create table test_table