On 22 July 2014 10:27, Andy Bradford <amb-fos...@bradfords.org> wrote: > Thus said "Andy Bradford" on 12 Jul 2014 13:36:58 -0600: > >> 2) The artifact rid was in the unclustered table, but when >> create_cluster() ran it prematurely removed it from the table. > > I have been able to successfully reproduce/cause this. When a large > number of artifacts are being transfered, they produce phantoms on the > server side of the sync operation. Eventually, the unclustered table > grows large enough that create_cluster() starts cleaning house and > building a new cluster artifact to replace all the entries in the > unclustered table. Then it deletes everything that it didn't just create > as part of creating clusters, including phantoms for which content has > not yet arrived (most importantly a checkin artifact in which the > manifest references a lot of other files). > > If no other artifacts reference the artifacts that were phantoms on the > unclustered table, now deleted, then the content seemingly disappears to > other clients that are trying to sync. If the content eventually gets > incorporated in other manifests then it will eventually sync because > they will discover the artifacts in those manifests, mark them as > phantoms and then request them with gimme cards. > > This is most easily reproduced by simply doing this in a clone of a > repository: > > $ jot 1500 | while read x; do dd if=/dev/urandom bs=1k count=1 | hexdump > > file.$x; done > $ fossil ci -m bigupdate --branch big > $ fossil up trunk > $ echo $RANDOM > file.1 > $ fossil ci -m back > > Now, never merge in the ``big'' branch and clients that have previously > cloned the server repository will never see the checkin to the branch > unless they use --verily. Otherwise, if the branch is merged into trunk > (or the checkin is edited and the branch closed), then suddenly it will > appear to those clients (assuming those changes don't get deleted from > the unclustered table first).
Thank you for the effort, this is or has been a long-standing issue indeed. I can't seem to reproduce what you describe - either that, or I'm missing the point (did you mean 'merge' as in 'fossil merge'?). I'm assuming you left out 'fossil add' (or 'addremove') twice in your example. I tried your example on a single host, hopefully to exclude complexity added by any physical network. (Do you think it's necessary to use 2 different hosts to reproduce the issue like you described?) I cloned using http:// before adding files, and then updated from within the cloned repo's workdir. (I can see the artifacts being received on the cloned repo's side, so I guess the attempt really ends there.) Longish typescript follows: --- michai@main:/fossils$ f ver This is fossil version 1.30 [619fa857c9] 2014-07-19 19:20:25 UTC michai@main:/fossils$ grep fossil /etc/inetd.conf http stream tcp nowait.1000 root /usr/local/bin/f /usr/local/bin/f http /fossils michai@main:/fossils$ f new --date-override 2014-01-01 ab.fossil project-id: e0b53c254d86b6445060df9c65a9017134b348db server-id: c72a02a0849d982ca8066b812521a1f7cde187af admin-user: michai (initial password is "794d90") michai@main:/fossils$ mkdir f michai@main:/fossils$ cd f michai@main:/fossils/f$ f open ../ab.fossil project-name: <unnamed> repository: /fossils/f/../ab.fossil local-root: /fossils/f/ config-db: /home/michai/.fossil project-code: e0b53c254d86b6445060df9c65a9017134b348db checkout: b58cc4d9818973107a8acba469dda6edd4ba9683 2014-01-01 00:00:00 UTC leaf: open tags: trunk comment: initial empty check-in (user: michai) checkins: 1 michai@main:/fossils/f$ pushd /tmp /tmp /fossils/f michai@main:/tmp$ mkdir f michai@main:/tmp$ cd f michai@main:/tmp/f$ f clone http://localhost/ab ab.fossil Round-trips: 1 Artifacts sent: 0 received: 0 Round-trips: 1 Artifacts sent: 0 received: 1 Round-trips: 2 Artifacts sent: 0 received: 1 Round-trips: 2 Artifacts sent: 0 received: 3 Clone finished with 461 bytes sent, 1155 bytes received Rebuilding repository meta-data... 0.0% complete... 100.0% complete... project-id: e0b53c254d86b6445060df9c65a9017134b348db server-id: 25482d35a0445e5710395d12423b421e90b9f4be admin-user: michai (password is "e4a7ef") michai@main:/tmp/f$ mkdir f michai@main:/tmp/f$ cd f michai@main:/tmp/f/f$ f open ../ab.fossil project-name: <unnamed> repository: /tmp/f/f/../ab.fossil local-root: /tmp/f/f/ config-db: /home/michai/.fossil project-code: e0b53c254d86b6445060df9c65a9017134b348db checkout: b58cc4d9818973107a8acba469dda6edd4ba9683 2014-01-01 00:00:00 UTC leaf: open tags: trunk comment: initial empty check-in (user: michai) checkins: 1 michai@main:/tmp/f/f$ popd /fossils/f michai@main:/fossils/f$ cat /tmp/f.sh #!/bin/sh jot 1500 | while read x; do dd if=/dev/urandom bs=1k count=1 | hexdump > file.$x; done f addr f ci -m bigupdate --branch big f up trunk echo $RANDOM > file.1 f add file.1 f ci -m back michai@main:/fossils/f$ /tmp/f.sh 1+0 records in 1+0 records out 1024 bytes transferred in 0.001 secs (1024000 bytes/sec) ... 1+0 records in 1+0 records out 1024 bytes transferred in 0.001 secs (1024000 bytes/sec) ADDED file.1 ADDED file.10 ... ADDED file.998 ADDED file.999 added 1500 files, deleted 0 files New_Version: 19fd1ac6dc46bf5e36b46d5e86bb4089670bc27c REMOVE file.1 REMOVE file.10 ... REMOVE file.998 REMOVE file.999 ------------------------------------------------------------------------------- updated-to: b58cc4d9818973107a8acba469dda6edd4ba9683 2014-01-01 00:00:00 UTC leaf: open tags: trunk comment: initial empty check-in (user: michai) changes: 1500 files modified. "fossil undo" is available to undo changes to the working checkout. ADDED file.1 New_Version: c5c46cb314a32ca998d02cb69361a7a2e099995d michai@main:/fossils/f$ f stat repository: /fossils/f/../ab.fossil local-root: /fossils/f/ config-db: /home/michai/.fossil checkout: c5c46cb314a32ca998d02cb69361a7a2e099995d 2014-07-22 10:22:24 UTC parent: b58cc4d9818973107a8acba469dda6edd4ba9683 2014-01-01 00:00:00 UTC leaf: open tags: trunk comment: back (user: michai) michai@main:/fossils/f$ pushd /tmp/f/f /tmp/f/f /fossils/f michai@main:/tmp/f/f$ f up Autosync: http://localhost/ab Round-trips: 1 Artifacts sent: 0 received: 0 Round-trips: 1 Artifacts sent: 0 received: 0 Round-trips: 2 Artifacts sent: 0 received: 0 Round-trips: 2 Artifacts sent: 0 received: 2 Round-trips: 3 Artifacts sent: 0 received: 2 Round-trips: 3 Artifacts sent: 0 received: 202 Round-trips: 4 Artifacts sent: 0 received: 202 Round-trips: 4 Artifacts sent: 0 received: 602 Round-trips: 5 Artifacts sent: 0 received: 602 Round-trips: 5 Artifacts sent: 0 received: 1402 Round-trips: 6 Artifacts sent: 0 received: 1402 Round-trips: 6 Artifacts sent: 0 received: 1505 Pull finished with 38451 bytes sent, 2429609 bytes received ADD file.1 ------------------------------------------------------------------------------- updated-to: c5c46cb314a32ca998d02cb69361a7a2e099995d 2014-07-22 10:22:24 UTC leaf: open tags: trunk comment: back (user: michai) changes: 1 file modified. "fossil undo" is available to undo changes to the working checkout. michai@main:/tmp/f/f$ f tim === 2014-07-22 === 10:22:24 [c5c46cb314] *CURRENT* back (user: michai tags: trunk) 10:22:22 [19fd1ac6dc] bigupdate (user: michai tags: big) === 2014-01-01 === 00:00:00 [b58cc4d981] *BRANCH* initial empty check-in (user: michai tags: trunk) +++ no more data (3) +++ michai@main:/tmp/f/f$ f bra big * trunk michai@main:/tmp/f/f$ f up big Autosync: http://localhost/ab Round-trips: 1 Artifacts sent: 0 received: 0 Round-trips: 1 Artifacts sent: 0 received: 0 Pull finished with 279 bytes sent, 315 bytes received UPDATE file.1 ADD file.10 ADD file.100 ... ADD file.998 ADD file.999 ------------------------------------------------------------------------------- updated-to: 19fd1ac6dc46bf5e36b46d5e86bb4089670bc27c 2014-07-22 10:22:22 UTC leaf: open tags: big comment: bigupdate (user: michai) changes: 1500 files modified. "fossil undo" is available to undo changes to the working checkout. michai@main:/tmp/f/f$ ls | wc -l 1500 michai@main:/tmp/f/f$ --- Michai _______________________________________________ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
