Re: What broke snapshots now?
On Sun, 2005-07-10 at 00:38 +0100, David Woodhouse wrote: Doh. I thought I'd already done that, but in fact that was for the scripts which feed the mailing list, while the snapshot script kept using my copy. Ok, the snapshot script starts working again if I change a few environment variables to match what the tools now expect. Now the mailing list feed isn't happy though -- it stopped being able to pull from your tree at around 0600 UTC (which I think is then the last DRM fix was added). I got this when trying to update... Tree change: 0109fd37046de64e8459f8c4f4706df9ac7cc82c:f179bc77d09b9087bfc559d0368bba350342ac76 error: cannot read sha1_file for ce68a60e5c503aaef0a98f8d754effb6c7d9ee99 fatal: unable to read destination tree (ce68a60e5c503aaef0a98f8d754effb6c7d9ee99) Applying changes... Fast-forwarding 0109fd37046de64e8459f8c4f4706df9ac7cc82c - f179bc77d09b9087bfc559d0368bba350342ac76 on top of 0109fd37046de64e8459f8c4f4706df9ac7cc82c... error: cannot read sha1_file for ce68a60e5c503aaef0a98f8d754effb6c7d9ee99 fatal: failed to unpack tree object f179bc77d09b9087bfc559d0368bba350342ac76 Since it's just a fast-forward, I just copied the 'origin' tag into the 'master' to move it forward. But it's still not happy: hera /home/dwmw2/git/mail-2.6 $ cg-diff -r 0109fd37046de64e8459f8c4f4706df9ac7cc82c:f179bc77d09b9087bfc559d0368bba350342ac76 error: cannot read sha1_file for ce68a60e5c503aaef0a98f8d754effb6c7d9ee99 fatal: unable to read destination tree (ce68a60e5c503aaef0a98f8d754effb6c7d9ee99) -- dwmw2 - To unsubscribe from this list: send the line unsubscribe git in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: What broke snapshots now?
On Sun, 10 Jul 2005, David Woodhouse wrote: Now the mailing list feed isn't happy though -- it stopped being able to pull from your tree at around 0600 UTC (which I think is then the last DRM fix was added). I got this when trying to update... Which script is this? I'm looking at your scripts, but cg-feedmaillist.sh is unreadable for me, so I can't see all of it. Anyway, it's possible that this is a temporary problem: one of the issues is that since you seem to be using the rsync: protocol for updating things, what happens is that if the mirroring is off a bit, you may have gotten a new head, but not all the objects. Then you'd get exactly this. I don't know how to solve that, really. It's fundamental to any mirroring that doesn't guarantee some kind of ordering, ie the dumb kind. All the native git pack-file synchronization protocols should be strictly ordered, ie they will unpack and write the new objects _before_ they write the new refs that point to them.. But you can just imagine what happens if the mirror script is busy mirroring at the same time, and has already mirrored the object directories before we wrote all of them, but then gets to the refs/ directory by the time we have written the new refs (and all the new object - but it didn't notice that because it had already mirrored the object dirs). See what I'm saying? The thing fixes itself after a while when you try again and the next round of mirroring has gotten the objects it missed the last time, but there fundamentally isn't anything to be done about the fact that the mirroring wasn't atomic. Only a git-aware mirror could do an atomic snapshot. I'm actually surprised that people haven't complained more about this than they have. I know it's happened a couple of times: it bit Dave Jones' scripts once, and it's caused webgit to sometimes report permission denied problems (or whatever) and then showing empty logs. The fact that my upload scripts _do_ upload in the right order, together with the fact that rsync normally syncs alphabetically (ie objects before refs) means that we _normally_ don't see this, but it's occasionally unavoidable when I happen to do a new push while a previous mirror sweep is going on. Of course, in this case it may also be just some cogito thing that doesn't like pack-files. The fact that it complains about an object that is not actually either of the heads of the diff makes me suspect it's really a missing object, though, which in turn makes me suspect the above mirror mis-feature. Linus - To unsubscribe from this list: send the line unsubscribe git in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: What broke snapshots now?
On Sun, 2005-07-10 at 10:08 -0700, Linus Torvalds wrote: Which script is this? I'm looking at your scripts, but cg-feedmaillist.sh is unreadable for me, so I can't see all of it. Hm. Dunno why that happened -- it's readable now, and also at http://david.woodhou.se/cg-feedmaillist.sh Anyway, it's possible that this is a temporary problem: one of the issues is that since you seem to be using the rsync: protocol for updating things, what happens is that if the mirroring is off a bit, you may have gotten a new head, but not all the objects. Then you'd get exactly this. It's done locally on hera though -- so the mirroring shouldn't be a problem. IIRC the reason it uses rsync is because I wasn't getting tags when it was using whatever other method was the default for a local 'parent repository'. That was actually more relevant for the snapshots than the mailing list feed, though -- so even if it isn't fixed now, I could live without tags. More usefully though, if ordering really isn't a problem on your repository then I should probably rewrite the script to work directly from that instead of from a copy. -- dwmw2 - To unsubscribe from this list: send the line unsubscribe git in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: What broke snapshots now?
On Sat, 9 Jul 2005, David Woodhouse wrote: Does git on master.kernel.org need to be updated to handle packed objects? See attached. Yes, looks that way. Except it's not git on master.kernel.org, it's git in your home directory, I suspect. I expressly held off packing the kernel repo until git had been updated on kernel.org. So while the regular git on kernel.org should already be up-to-date, you have PATH=/home/dwmw2/cogito:/usr/bin:/bin in your script ;) Linus, please could you add the snapshot script to your regression testing? http://david.woodhou.se/git-snapshot.sh None of the git-specific parts should have broken - you don't even do anything strange there, I think we already have regression tests in git for everything you use. But the fact that you had an old version is not something I can do much about ;) Linus - To unsubscribe from this list: send the line unsubscribe git in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: What broke snapshots now?
On Sat, 2005-07-09 at 09:15 -0700, Linus Torvalds wrote: Yes, looks that way. Except it's not git on master.kernel.org, it's git in your home directory, I suspect. I expressly held off packing the kernel repo until git had been updated on kernel.org. Doh. I thought I'd already done that, but in fact that was for the scripts which feed the mailing list, while the snapshot script kept using my copy. I've moved it out of the way now; sorry for the noise. -- dwmw2 - To unsubscribe from this list: send the line unsubscribe git in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
