The GitHub Actions job "Validate and test" on pekko-persistence-r2dbc.git/main has failed. Run started by GitHub user ptrdom (triggered by ptrdom).
Head commit for run: 5a212bab7358e77e70f5d53d666f481b9758d2ca / PJ Fanning <[email protected]> core: snapshots as starting points for eventsBySlices and eventsByPersistenceId (#515) * core: snapshots as starting points for eventsBySlices and eventsByPersistenceId Port of akka/akka-persistence-r2dbc#410, #412, #438, #464 and #469 (https://github.com/akka/akka-persistence-r2dbc/pull/410, https://github.com/akka/akka-persistence-r2dbc/pull/412, https://github.com/akka/akka-persistence-r2dbc/pull/438, https://github.com/akka/akka-persistence-r2dbc/pull/464, https://github.com/akka/akka-persistence-r2dbc/pull/469), from the akka-persistence-r2dbc 1.2.0 release, which is available under the Apache License, Version 2.0 as of 2026-10-24. Motivation: Pekko core defines EventsBySliceStartingFromSnapshotsQuery, EventsByPersistenceIdStartingFromSnapshotQuery and their current variants, but the R2DBC read journal does not implement them. Consumers that start from zero or are far behind must replay all events. Modification: - Implement the four queries in the scaladsl and javadsl read journals, using StartingFromSnapshotStage to emit snapshots first, then the events after them. - SnapshotDao implements BySliceQuery.Dao for Postgres, Yugabyte and MySQL, and stores the db_timestamp and tags of the corresponding event when start-from-snapshot is enabled. - New config query.start-from-snapshot.enabled (default off), with snapshot.start-from-snapshot.enabled following it. Disabled, the snapshot plugin does not use the new columns. - Snapshot envelopes have source "SN"; rows carry their own source. - The migration tool stores the timestamp and tags of the migrated event with migrated snapshots when enabled. - DDL: db_timestamp and tags columns and snapshot_slice_idx for all dialects, plus migration scripts in ddl-scripts/migration. - Docs: query, snapshot and migration guide sections. Result: Projections can start from snapshots instead of replaying all events. Tests: - New StartingFromSnapshotStageSpec, EventsBySliceStartingFromSnapshotSpec, EventsByPersistenceIdStartingFromSnapshotSpec (ported, plus javadsl and snapshot-source tests), StartFromSnapshotDisabledSpec and a MigrationToolSpec test - PostgreSQL: sbt core/test 391 succeeded; sbt migration/test 8 succeeded - MySQL: snapshot-related specs 44 succeeded; DurableStateBySliceSpec fails locally on upstream/main as well (passes in CI) - Yugabyte: not run locally, left to CI - Migration scripts verified on the old schema for PostgreSQL and MySQL - sbt core/mimaReportBinaryIssues migration/mimaReportBinaryIssues: no issues References: Refs akka/akka-persistence-r2dbc#410, #412, #438, #464, #469 * core: support snapshots as starting points with H2 Motivation: The H2 support (#517) was merged after this change was written. H2SnapshotDao overrode the upsertSql that is now built by createUpsertSql, the H2 snapshot table had no db_timestamp and tags columns, and the snapshot slice queries use Postgres-specific SQL. Modification: - H2SnapshotDao overrides createUpsertSql with MERGE statements, with and without db_timestamp and tags, reads tags with H2Utils, uses H2 SQL for the bucket counts and the app timestamp as the current db timestamp, like H2QueryDao. - The H2 snapshot table has the db_timestamp and tags columns. The snapshot slice index is created when create-slice-indexes and start-from-snapshot are enabled, since an existing table may not have the db_timestamp column otherwise. - The snapshot settings of the query plugin have an empty h2.additional-init. The query creates the snapshot table, like it creates the journal table, and the additional-init belongs to the snapshot plugin. Result: The queries that use snapshots as starting points work with H2. Tests: - sbt -Dpekko.persistence.r2dbc.dialect=h2 "core/testOnly org.apache.pekko.persistence.r2dbc.internal.h2.* org.apache.pekko.persistence.r2dbc.snapshot.* org.apache.pekko.persistence.r2dbc.query.*": before this change core does not compile; with only the H2SnapshotDao and DDL changes, the StartingFromSnapshot specs fail ("No configuration setting found for key 'h2'") and StartFromSnapshotDisabledSpec fails (Column "DB_TIMESTAMP" not found); after it, 112 succeeded, 0 failed - H2TableCreationSpec: new test stores and loads db_timestamp and tags, and checks the snapshot slice index - sbt -Dpekko.persistence.r2dbc.dialect=h2 core/test: 339 succeeded, 0 failed - sbt core/mimaReportBinaryIssues: no issues References: Follows #517 * core: don't miss events when starting from snapshots Motivation: Review of #515 found two bugs in the slice queries that start from snapshots, both inherited from akka-persistence-r2dbc 1.2.0: - rows dropped by the snapshot filter did not update the query state, so a fully filtered page completed the current query early and stalled the live query - with NoOffset the event query started at the min snapshot timestamp, so older events of entities without a snapshot were missed Modification: - BySliceQuery applies the filter after ContinuousQuery, on the envelopes, so filtered envelopes still advance the offset and the row count - The event query always starts at the given offset - Document that the events are still read from the offset, and that the offsets of the emitted envelopes are not increasing - Log a warning when the event of a snapshot is not found, and document the snapshot settings that must be set for custom plugin ids - The migration updates skip deleted events, like the snapshot plugin, and the guide mentions batching and that the feature is new in 2.0.0 Result: Events after a full page of filtered events, and events of entities without a snapshot, are emitted. Tests: - New EventsBySliceStartingFromSnapshotSmallBufferSpec (buffer-size 10) and an EventsBySliceStartingFromSnapshotSpec test for an entity without a snapshot: all 4 fail without the fix on H2 - H2: query and snapshot specs 109 succeeded, 0 failed - PostgreSQL: sbt core/test 429 succeeded; sbt migration/test 8 succeeded - Postgres migration script checked against a deleted event - MySQL and Yugabyte: left to CI - sbt core/mimaReportBinaryIssues migration/mimaReportBinaryIssues: no issues References: Refs #515 * core: apply scalafmt * core: support snapshots as starting points with MariaDB Motivation: After rebasing onto #534, MariaDBSnapshotDao overrode upsertSql, which is now private, so core did not compile. Its upsert also lacked the db_timestamp and tags columns used when start-from-snapshot is enabled. Modification: - MariaDBSnapshotDao overrides createUpsertSql with both variants, using VALUES(col) instead of the MySQL row alias - StartFromSnapshotDisabledSpec creates the legacy table with BLOB payloads for MariaDB - The migration guide lists the MySQL script for MariaDB too Result: The start-from-snapshot queries can be used with the mariadb dialect. Tests: - sbt core/Test/compile migration/Test/compile - MariaDB not run locally; relying on the CI MariaDB job References: Refs #515, #534 * core: fall back to the write timestamp in the snapshot migration scripts Motivation: The start-from-snapshot migration scripts only set db_timestamp from a non-deleted event with the snapshot's seqNr. Snapshots whose event was replaced by a delete marker, or deleted past the snapshot, kept a NULL db_timestamp and were never emitted by the queries that start from snapshots. The scripts were also not run by any test or CI job. Modification: - The Postgres, Yugabyte and MySQL scripts set db_timestamp from write_timestamp for the remaining snapshots, like the snapshot plugin does when it doesn't find the event - The migration guide shows and explains the second update - New StartFromSnapshotMigrationScriptSpec runs the dialect's script on a legacy snapshot table and checks that the queries emit the migrated snapshots, with the event's timestamp and tags when the event exists Result: Migrated snapshots without a matching event are emitted by the queries that start from snapshots, and the scripts are tested on every database. Tests: - StartFromSnapshotMigrationScriptSpec and StartFromSnapshotDisabledSpec pass on PostgreSQL, MySQL, MariaDB and Yugabyte (docker); the new spec is cancelled on H2, which has no migration script - Without the script change the new spec fails on PostgreSQL and MySQL, only the snapshot with an existing event is emitted - sbt ++3.3 core/Test/compile, scalafmtCheckAll, headerCheckAll References: Refs #515 * core: apply scalafmt to StartFromSnapshotMigrationScriptSpec * core: don't depend on the entity type counter in persistence ids specs Motivation: The Postgres and Yugabyte CI jobs failed in the event sourced CurrentPersistenceIdsQuerySpec. The persistence ids are returned in string order, and the specs expected the order in which their entity types were created. With the new StartFromSnapshotMigrationScriptSpec the shared counter reached TestEntity-100, which sorts before TestEntity-96. Modification: Sort the expected ids in the event sourced and durable state CurrentPersistenceIdsQuerySpec, and take the afterId of the entity type test from that entity type's ids. Result: The specs pass whatever the value of the entity type counter. Tests: - With the TestData counter starting at 95, both specs failed on H2 before the change and pass after it - Both specs pass on H2 with the counter starting at 0 References: Refs #515 --------- Co-authored-by: Domantas Petrauskas <[email protected]> Report URL: https://github.com/apache/pekko-persistence-r2dbc/actions/runs/41376329416 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
