> > > 1. a post-commit error "database is locked" > > > 2. new representations will not be added in the rep-cache.db > > > 3. deduplication does not work for new data committed at this time > > > 4. commits work with delays.
> > I look to hearing Denis's concerns with the sharding approach. > > In addition, if we ever need atomic operations on the entire > rep-cache, we will have to use ATTACH DATABASE statement [1] with the master > journal [2], which I think is not used anywhere now, and is not supported by > all journaling modes. > Fair point. Let's see the pros and cons: - pro: fixes issues #1 through #4, and the "large commits starve the following commit" issue as well - con: if someday we have a need for cross-shard atomic transactions, we'll have to either (1) forbid rep-cache.db shards from using WAL mode; or (2) figure out another way to solve whichever problem we would use atomic cross-shard transactions to solve if we had an unsharded rep-cache.db; or (3) revert to a monolithic rep-cache.db and figure out another way to solve the problems this thread is about. > - If I'm not mistaken, this approach requires a format bump. So this does not > fix the problem for existing repositories. It is also necessary to perform > the division into shards in some form, which means that a fast in-place > upgrade also probably will not fix the problem. An in-place upgrade would be straightforward. > I've snipped most of your points as I already answered them upthread. If I didn't respond to some point which _hasn't_ already been addressed upthread, just ask it again. My concerns stand. Daniel
