nickva commented on issue #3675:
URL: https://github.com/apache/couchdb/issues/3675#issuecomment-884397448
I think it might be doable if we record a few more bits in the checkpoint
documents and change their shape a bit.
* That's true that we don't record target sequences in checkpoints [1]. We
could add that and bump the checkpoint version format.
* I think we'd need uniquely identify endpoint instances by a UUID. Then,
we could add a new type of checkpoints, in addition to the current ones, which
might look like `_local/rep_checkpoint_$fromid_$toid`. In `main`, we recently
added such per-db instance "uuid" field[2] already! Now wondering if it would
this be possible to add it to 3.x dbs too. A new field the _dbs docs perhaps...?
* To have second-order transitive checks, we may have to add a step during
checkpointing where the replication jobs would copy all the
`_local/rep_checkpoint_$fromid_$toid` checkpoint docs from the source to the
target. So when `us-west` -> `store1` is initiated, we'd look for common
`$fromid` -...-> `store1` and `$fromid` -...-> `us-west` checkpoints and find
`us-east` -> `store1` (on `store1` target) and `us-east` -> `us-west` (on
`us-west` source).
* One thing I am not 100% sure on is if we could always compare sequences
to pick the max start sequence. On `main` `update_seq`s are FDB versionstamps,
on 3.x we have shard nodes and uuids in there. We could have case of a random
graph with PouchDb, `main` and `3.x` clustered CouchDB endpoints and such.
[1] example checkpoint:
```json
{
"_id": "_local/d99e532c1129e9cacbf7ed085deca509",
"_rev": "0-17",
"history": [
{
"doc_write_failures": 0,
"docs_read": 249,
"docs_written": 249,
"end_last_seq":
"249-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE5tygQLsZiaGqWlpxpjKcRqRxwIkGRqA1H-oSeVgk5JMkkxNkg0xdWUBAJ5nJWc",
"end_time": "Wed, 21 Jul 2021 17:10:06 GMT",
"missing_checked": 253,
"missing_found": 249,
"recorded_seq":
"249-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE5tygQLsZiaGqWlpxpjKcRqRxwIkGRqA1H-oSeVgk5JMkkxNkg0xdWUBAJ5nJWc",
"session_id": "dc645ae85a7c3fe6c3ac5da8e73077ce",
"start_last_seq":
"228-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE0tzgQLsZiaGqWlpxpjKcRqRxwIkGRqA1H-oSflgk5JMkkxNkg0xdWUBAJgFJVI",
"start_time": "Wed, 21 Jul 2021 17:01:21 GMT"
},
...
],
"replication_id_version": 4,
"session_id": "dc645ae85a7c3fe6c3ac5da8e73077ce",
"source_last_seq":
"249-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE5tygQLsZiaGqWlpxpjKcRqRxwIkGRqA1H-oSeVgk5JMkkxNkg0xdWUBAJ5nJWc"
}
```
[2] Unique, per db-instance UUID on `main`
```json
http $DB/mydb1
{
...
"instance_start_time": "0",
"sizes": {
"external": 34,
"views": 0
},
"update_seq": "00000008d5c93d5a00000000",
"uuid": "ce0279e40045b4f7cd6cd4f60ffd3b3c"
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
