That’s a good point. CASSANDRA-12106 would let you blacklist those partitions if you know they’re invalid. That’s probably strictly easier than truncating the table.
> On Jul 15, 2026, at 9:01 AM, Abe Ratnofsky <[email protected]> wrote: > > Read repair will propagate the future-writes, so if you scrub them from one > replica and have concurrent reads, they can re-appear. > > Depending on your requirements you might also be able to post-filter the > future-writes at query time, by including SELECT WRITETIME(col) in your read > queries and skipping them. > > On Wed, Jul 15, 2026, at 8:39 AM, Jeff Jirsa wrote: >> Double writing / double reading gets messy because it requires an awful >> lot of work to ensure it’s correct as you move data around. >> >> Just fixing it during compaction or scrub doesnt require app changes, >> just disable repair, start rewriting the sstables, enable repair again >> when you’re done. >> >> Especially if 18352 gives you a cassandra-internal “is this a legal >> timestamp” predicate, that seems way easier than changing all the app >> logic. >> >> >>> On Jul 15, 2026, at 8:21 AM, Abe Ratnofsky <[email protected]> wrote: >>> >>> For recovery, I’d recommend approaching this as a regular table migration, >>> where you start writing to a new table and backfill using >>> cassandra-analytics, filtering out all cells with WRITETIME in the distant >>> future. >>> >>> Freeze your backups so they don’t get purged. You might be able to recover >>> some overwritten values from backup SSTables if they weren’t compacted yet. >>> >>> On Wed, Jul 15, 2026, at 8:16 AM, Jeff Jirsa wrote: >>>> It’s not that it’s not running, it’s that ntpd in startup phases can >>>> have big steps in some configs. Also alert on large clock drift. If you >>>> run cassandra, you need alerts on ntpd >>>> >>>>> On Jul 15, 2026, at 8:13 AM, Yakir Gibraltar <[email protected]> wrote: >>>>> >>>>> Configure Cassandra to start after ntpd service , easy on systemd. >>>>> >>>>> Thank you, Yakir Gibraltar >>>>> >>>>> On Wed, Jul 15, 2026, 10:47 Panagiotis Melidis via user >>>>> <[email protected]> wrote: >>>>>> Hi all,____ >>>>>> __ __ >>>>>> I'm looking for guidance on a scenario involving NTP clock skew and its >>>>>> impact on Cassandra data.____ >>>>>> __ __ >>>>>> If an NTP malfunction causes the system clock on multiple Cassandra >>>>>> nodes to jump far into the future (e.g., 100 years), any writes >>>>>> processed during that window will carry far-future timestamps. Once the >>>>>> clock is corrected, these entries effectively become permanent and >>>>>> shadow any new write/delete in the affected partition keys.____ >>>>>> __ __ >>>>>> The only viable recovery approach at scale I've been able to identify is >>>>>> truncating the affected tables.____ >>>>>> 1. Are there any other known viable recovery procedures that don't >>>>>> require a full truncate?____ >>>>>> 2. Are there any known existing or planned preventive measures in >>>>>> Cassandra to guard against such scenarios?____ >>>>>> __ __ >>>>>> Any pointers to prior discussions, tooling, or patches would be greatly >>>>>> appreciated.____ >>>>>> __ __ >>>>>> Thanks.____
