Hello
eventsByTag in the CassandraReadJournal uses a materialized view to read
the events. Currently materialized view is created by
CREATE MATERIALIZED VIEW IF NOT EXISTS $eventsByTagViewName$tagId AS
SELECT tag$tagId, timebucket, timestamp, persistence_id, partition_nr,
sequence_nr, writer_uuid, ser_id, ser_manifest, event_manifest, event, message
FROM $tableName
WHERE persistence_id IS NOT NULL AND partition_nr IS NOT NULL AND
sequence_nr IS NOT NULL
AND tag$tagId IS NOT NULL AND timestamp IS NOT NULL AND timebucket IS NOT
NULL
PRIMARY KEY ((tag$tagId, timebucket), timestamp, persistence_id,
partition_nr, sequence_nr)
WITH CLUSTERING ORDER BY (timestamp ASC)
"""
Partition key is (tag$tagId, timebucket) where timebucket has the following
format: DateTimeFormatter.ofPattern("yyyyMMdd")
I've got a huge amount of events with the same tag. As a result all events
are stored on single cassandra node for one day, since all nodes
participated in writing events, this "materialized view" node slows down
the whole system.
Possible workarounds:
1.Use set of tagId instead of one, to calculate tagId simple hash function
can be used: hash(event) % m. But in this case it may slow down query for
read part, as it should find every event across all nodes in the cluster.
2. Also I thought about counter-based solution, change tag Id every 10k
events
3. contribute to cassandra plugin to make timebucket configurable and add
minutes to the pattern.
Is it make sense? I've got some doubts :) . Because, anyway, one of the
nodes will "suffer" from materialized view and slow down the whole system.
I'll be glad to hear any thoughts about it
Thanks,
Serhii
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
