Re: Internal Handling of Map Updates

2016-06-02 Thread Eric Stevens
If it's overwrites and append only with no removes, an UPDATE will let you do that to standard collections. Like INSERT, UPDATE acts like an UPSERT. On Thu, Jun 2, 2016, 12:52 AM Matthias Niehoff < matthias.nieh...@codecentric.de> wrote: > JSON would be an option, yes. A frozen collection would

Re: Internal Handling of Map Updates

2016-06-02 Thread Matthias Niehoff
JSON would be an option, yes. A frozen collection would not work for us, as the updates are both overwrites of existing values and appends of new values (but never a remove of values). So we end up with 3 options: 1. use clustering columns 2. use json 3. save the row not using the

Re: Internal Handling of Map Updates

2016-06-01 Thread Eric Stevens
>From that perspective, you could also use a frozen collection which takes away the ability to append, but for which overwrites shouldn't generate a tombstone. On Wed, Jun 1, 2016, 5:54 PM kurt Greaves wrote: > Is there anything stopping you from using JSON instead of a

Re: Internal Handling of Map Updates

2016-06-01 Thread kurt Greaves
Is there anything stopping you from using JSON instead of a collection? On 27 May 2016 at 15:20, Eric Stevens wrote: > If you aren't removing elements from the map, you should instead be able > to use an UPDATE statement and append the map. It will have the same effect > as

Re: Internal Handling of Map Updates

2016-05-27 Thread Eric Stevens
If you aren't removing elements from the map, you should instead be able to use an UPDATE statement and append the map. It will have the same effect as overwriting it, because all the new keys will take precedence over the existing keys. But it'll happen without generating a tombstone first. If

Re: Internal Handling of Map Updates

2016-05-27 Thread Matthias Niehoff
We are processing events in Spark and store the resulting entries (containing a map) in Cassandra. The results can be new (no entry for this key in Cassandra) or an Update (there is already an entry with this key in Cassandra). We use the spark-cassandra-connector to store the data in Cassandra.

Re: Internal Handling of Map Updates

2016-05-25 Thread kurt Greaves
Literally just encountered this exact same thing. I couldn't find anything in the official docs related to this but there is at least this blog that explains it: http://www.jsravn.com/2015/05/13/cassandra-tombstones-collections.html and this entry in ScyllaDB's documentation:

Re: Internal Handling of Map Updates

2016-05-25 Thread Tyler Hobbs
If you replace an entire collection, whether it's a map, set, or list, a range tombstone will be inserted followed by the new collection. If you only update a single element, no tombstones are generated. On Wed, May 25, 2016 at 9:48 AM, Matthias Niehoff < matthias.nieh...@codecentric.de> wrote:

Internal Handling of Map Updates

2016-05-25 Thread Matthias Niehoff
Hi, we have a table with a Map Field. We do not delete anything in this table, but to updates on the values including the Map Field (most of the time a new value for an existing key, Rarely adding new keys). We now encounter a huge amount of thumbstones for this Table. We used sstable2json to